This is the fourth part in a series profiling the Cloud -Native Journey. This part discusses what an IT department, that has transformed into a Cloud-Native enterprise, looks like and how it got there. First, this piece points out the fundamental importance of culture and change. We highlight some research on how IT is doing to support the business. Then, we get into programming the meatware to effectively run inside a Cloud-Native organization.
It also fills in some of the gaps left by the first two parts, covering the “greenfield” (new applications) and “legacy” (working with existing application) journeys. In case you are wondering, the first part gave an introductory overview of all journeys and the “why” for going on a Cloud-Native journey in the first place.
Beyond Tactics: Transforming To A More Resilient Culture
The transformation part of the Cloud-Native Journey is largely about how you establish the new behavior and habits of a software defined business. While many of the tactics and technologies I covered in the greenfield and legacy parts carry over here, the transformation journey is focused on building new operating model for IT. Of course, this means changes to responsibilities, processes, and technology. Yet, there is an even more critical foundation—creating and cultivating a beneficial culture. This can seem like “soft” concerns, but just as zombie movies are never really about zombies and instead about human culture, new technologies are rarely about the technology themselves but new ways of using technology. Of course, zombies, and technologies, are more than just MacGuffins and play a major role themselves, but focusing on the human, behavioral, and cultural element is the transformative part.
In our case, the MacGuffin, as covered in the introduction to this series, is the ability to reliably and safely deploy software weekly, if not multiple times a day. Importantly, this addresses a larger goal—you get the ability to start investing more into digital customer experiences or new, digital business models by more perfectly designing and crafting software. The cultural change starts with these goal in mind—leaping at the opportunity to improve your software and more closely aligning your applications with business strategy. To make this change sustainable, you’ll need to change how the IT department is organized and run. The new roles, responsibilities, practices and processes you’ll put in place, intermixed with new group behaviors, will become your new culture.
What You’re Doing Isn’t Working
Companies definitely feel the need to improve the role IT plays in the overall business, as multiple studies have been showing. One of the more illustrative studies comes from the Cutter Consortium and compares the value of IT to business innovation over the past three years (the actual survey question is posed at the bottom of the chart):
The drop in IT being a key enabler for business innovation, from 54% in 2013 to 31% in 2015, is actually shocking. It points to a real need—for leaders in IT to focus on transforming their organization into a strategic business partner.
Let’s take a look at some of the ways IT starts transforming and hopefully reverses this downward trend in usefulness. Many of the practices we’ve already covered are still highly applicable—for example—managing your IT portfolio to free up time to work on new projects, being as strict about test coverage as possible, and using a cloud platform. This last part will expand on some of those concepts and cover ongoing culture cultivation, ensuring you can reap the greatest benefits from being a Cloud-Native enterprise.
Your Meat is Rotting: Meatware is the Problem
As I talk with companies who are looking to transform how they do IT, I realize more and more that the problem is not so much technology, or even how that technology is used. The problem is the unhelpful processes, behavior, and culture that is in place. You can think of these as thought technologies, but I like to call them meatware. Much of what it takes—to transform IT organizations into centers for innovation—is about changing the IT culture from a command and control, big planning up-front, organized by function (or silo’d) mind-set. Most IT departments operate in this traditional way, creating teams aligned by function and capability then assigning them to projects as needed. When the problems of IT are well understood, and the IT team’s job is to keep all the processes humming and perform routine maintenance, this kind of approach can be good. However, when your task is exploring new processes and software products—as a Cloud-Native enterprise, reinventing and evolving how its business operates—this traditional, keep the trains running on time mindset is harmful.
There are many ways to look at these two modes of operating: bimodal IT, the three horizons, Agile vs. Six Sigma, etc. Out of all the great strategy framings, I like the explore/exploit model. When businesses start off, they are exploring new innovations in products and services as well as how they create and deliver those wares to customers. This process requires experimenting, learning, and risk taking until a team finally finds the right mix of customer need, product, and go-to-market model, ultimately achieving competitive advantage and exploiting a growing, profitable, business model.
Once a company finds that model, they usually switch to the exploit mode of operating. This mode knows exactly what the business is, understands the customer need, has a product or service that meets that need, and understands how to effectively get that offering into a customer’s hand in exchange for money. With organizations like governments and nonprofits, the end goal is satisfying “the mission,” fundraising, or providing a service to citizens. In the exploit mode, companies look to optimize how their processes run and “execute to plan” because the plan is working!
Too often, IT departments who need to be operating in an explore mind-set have a culture of operating in an exploit mindset—they fail to change their culture over to the needed explore mode. It’s little wonder then that Gartner is predicting 90% failure rate in DevOps adoption if organizations don’t address cultural issues. The first step, then, is to become cognizant of which mode you should be operating in. This knowledge—like all good strategy—needs to be spread down the ranks to individuals and become a key enabler of transforming your IT department. Here is how one Pivotal customer explains the transformation:
CoreLogic brought software engineers to Pivotal Labs to learn Pivotal’s extreme agile software development methodology. The methodology focuses on test-driven development, pair programming, short development cycles, and continuous verification and integration of code to improve software quality and flexibility and reduce cost. As a result, the culture of CoreLogic product management and technology groups has fundamentally changed to become more nimble and collaborative.
“For us, it’s been much more than a technological transformation moving to a (Cloud-Native Platform). It’s a new way to develop products,” says Leurig. “It’s the most exciting thing we’ve done in the last 12 months.”
Strategy & Motivation
Instead of legacy meatware, Cloud-Native enterprises are typically comprised of self-motivated and directed teams. This reduces the amount of time it takes to make decisions, deploy code, and see if the results helped move the needle. More than just focusing on speed of decision making and execution, building up these intrinsically motivated teams helps spark creativity, fueling innovative thinking. Instead of being motivated by metrics like number of tickets closed or number of stories/points implemented, these teams are motivated by ideas like increased customer satisfaction with faster forms processing or helped people track and improve their health by moving the application to where the people are when they’re being healthy.
In my experience, one of the first steps in shifting how your people think—from the Pavlovian KPI bell—is to clearly explain your strategy and corporate principals. Having worked in strategy roles in the past, I’ve learned how poorly companies articulate their goals, constraints, and strategy. While there are many beautiful (and not so beautiful!) presentations that extol a company’s top motivations and finely worded strategies, most employees are left wondering how they can help day to day.
Whether you’re a leader or an individual contributor, you need to know the actionable details of the overall strategy and goals. Knowing your company’s strategy, and how it will implement that strategy, is not only necessary for breeding self-motivated and self-managed people, but it also comes in handy when you’re looking to apply agile and lean principles to your overall continuous delivery pipeline. Tactically, this means taking the time to create detailed maps of how your company executes its strategy. For example, you might do value-stream mapping, alignment maps, or something like value chain mapping. This is a case where I, again, find that companies have much less than they think they do—often, organizations have piles of diagrams and documents—but—very rarely have something that illustrates everything—from having an idea for a feature to getting it in customer’s hands. A Cloud-Native enterprise will always seek to be learning from and improving that end-to-end process. So, it’s required to map your entire business process out and have everyone understand it. Then, we all know how we deliver value.
The process of value-stream mapping, for example, can be valuable for simply finding waste (so much so that the authors of Learning to See cite only focusing on waste removal as a lean anti-pattern). As one anecdote goes—after working for several weeks to finally get everything up on the big white board, one of the executives looked up at all the steps and time wasted in their process to get software out the door and said, “there’s a whole lot of stupid up there.” The goal of these exercises is not only removing stupid, but also to focus on and improve the processes.
Management Creates the Game…So, Hopefully They Show Up
Coupled with discovery and experimentation of strategy and approach, there is a necessary shift in the role of management. In an exploit mode of operating, management is often responsible for making decisions—approving changes, putting tops-down plans in place, and endlessly reviewing “status.” In a explore mode, management needs to shift to being more hands on, more like a personal trainer or coach that encourages exploration before quick decisions and immediate action.
Having grown up like many computer nerds, I played role playing games like Dungeons and Dragons extensively in youth. In that game, there are players who go on the adventure and a dungeon master who orchestrates the adventure, including narrating the adventure, controlling the monsters, doling out rewards, and enforcing (or not) the rules of the game. Without players to star in the game and a dungeon master to run the game, there is no game. Similarly, the role of management in a Cloud- Native enterprise is to create and orchestrate the game that the organization is playing. This is much different than sitting through endless meetings reviewing statuses and rewarding or punishing those responsible.
First, management needs to create the game by establishing the strategy, goals, and general operating constraints (you could say “principals,” but I prefer the Meat Loaf approach of specifying the small set of things you won’t do, assuming that anything else is possible). As put in Leading the Transformation:
Management needs to establish strategic objectives that make sense and that can be used to drive plans and track progress at the enterprise level. These should include key deliverables for the business and process changes for improving the effectiveness of the organization.
In addition to this game setting, the hands-on approach comes down to roaming the halls and participating in the process—especially at first—as teams are learning to become self-reliant, self-motivated, and self-managed. Here, leaders can lead by example and start to move out from behind the bustles of PowerPoint. Also, for more on how management’s role and tactics change in during this transformation, check out Siobhan McFeeney’s recent blog post on the topic.
Speaking of being more hands on, let’s look at a buffet of management tactics to start considering and deploying—starting with how to manage your IT portfolio.
Crafting the Organization: Products Running On A Platform
The traditional way we think about structuring the IT department is different than how Cloud- Native IT departments structure themselves. To massively simplify it, traditional IT departments are oriented around working on projects, where-as technology companies are oriented around working on products.
Traditional thinking tends to focus on service delivery, responding to requests as they come in. Granted, the request may be a large project, but the idea is to have the requester more or less know what they want. The IT department delivers on that and then keeps the service up and running. This is great for the exploit mode of thinking where requesters know what they want and don’t change requirements around too much. However, there are drawbacks to the exploit mind-set. First, it can lead to over-thinking each problem, often called analysis paralysis or solutionizing. In essence, you can end up doing both too much planning and then over-service (doing too much!) to solve an otherwise simple problem. However, there is a big problem with this approach. It is not always appropriate for the type of problem being solved. When the nature of problem is largely unknown, a project mind-set often results in poor solutions.
An innovative, product centric approach is what’s needed in the explore mode—where people don’t know what they want and are often (at first) completely wrong about what they want. Teams in this setting should be more closely attached to the software being written rather than parachuted in as needed. The team’s understanding of the problem being solved, approaches tried in the past, and the overall tribal knowledge will be invaluable in figuring out the right product to build. These integrated teams are not only important for product management continuity but also for ensuring that the product is resilient in production. As outlined back in the greenfield journey post, these teams needs to be kept small small—somewhere between 6 and 12 people—and you want them to have all the skills needed to for the full life-cycle of the product, including development, testing, design, and operations.
When it comes to operations, this is where DevOps plays an enabling role in Cloud-Native enterprises. If the team that wrote the application code is also responsible for keeping it up in production, that team will make sure they write resilient code. The team will seek to add operations staff directly to the team rather than leaving that as “someone else’s problem.” Setting up an organization like this requires, not only developers and operators comingled on teams, but creating an organization that supports the cloud platform. This introduces two new roles in IT, the platform operators and the platform developers.
Platform operators are responsible for keeping the cloud platform up and running along with updating the software and infrastructure. Early on, they may be responsible for consulting with the application teams as the organization learns Cloud-Native development and operations practices.
Platform developers work on the evolution of the cloud platform itself. They focus on adding in new services like databases, integrations with partners and third parties, and otherwise adding in industry and business specific business logic to the cloud platform. These platform developers are creating a product, one that’s targeted at the company’s developer teams.
As the illustration above shows, the amount of staff in each of these layers reduces as you go “down” the stack. This is because you’re focused on applications as the primary point of customer value, and the most staff are there, exploring and perfecting your software. Next down, especially at first, the platform developers will be a smaller pool but well staffed. The platform operators, in practice, tend to be a shockingly small number—from single digits inr smaller organizations to 10’s or 20’s in very large organizations. Finally, if you’re running all of this on your own— as many Pivotal customers do— you’ll need staff to take care of the raw infrastructure, from the hardware all the way down to the dirt under the datacenter. This is based on the early days of the Cloud-Native approach and may evolve, but, so far, this is what has been panning out in organizations that I’ve observed.
Melt The Special Snowflakes: Portfolio Consolidation
To transform, you’ll likely need to start consolidating your applications down as much as possible. If you’re like many of the large IT shops I see, you’ve accumulated 1,000’s of applications, many of which are one-offs and special snowflakes. In addition to heterogeneity, the second problem with these applications is that they’ve generated a lot of technical debt—compromises made to get them out the door and do them cheaply, including a lack automated test coverage. You’re sitting in a house of well reinforced paper mache, essentially. To address, most companies begin to systematically put in a program to reduce this debt. This takes many years to do, and you should be mindfully managing and measuring it to ensure that this debt doesn’t consume all of your resources. This is the continued portfolio management I discussed in the legacy part, with the addition of looking to consolidate some applications.
In addition to finding duplication in the portfolio and combining applications as it makes sense, the Cloud-Native approach seeks to move as many applications as possible to a shared cloud platform, like Pivotal Cloud Foundry. This will have tremendous resource scaling effects based on reducing variability in your portfolio, which will lower many costs while providing additional value:
- More automation and standardized management. Operations manages one platform and uses similar, if not the same, tools to monitor applications in production. Companies have long benefited from standardizing on operating systems or even building standard virtual machine images. This has still left plenty of variability for how applications were packaged, provisioned, updated, and managed in production. It’s only recently, with the advent of cloud platforms, that this next layer—the application layer—has been helpfully standardized with respect to automation and management in production.
- Standardized application lifecycle management (ALM). While development teams are free to choose languages, frameworks, and even the develop tools they use, each development team is working within the same contract specified by the cloud platform. This gives developers the freedom to choose how they code their applications, but limits the middleware and systems they can choose, reducing the overall effort to manage your portfolio of applications.
- Compliance and regulation speedup. By standardizing on the same underlying platform, you can greatly reduce the amount of paperwork needed for governance, risk, and compliance (GRC), as well as audits. As an example, the GSA has been able to reduce the length of time required to pass strict government audits for software releases down from 9 months to just days by standardizing on a shared cloud platform.
Always be on the look-out for consolidating services and applications. At scale, this is how you’ll survive. The trick is make sure you’re not giving up flexibility and speed in favor of controls. In my experiences, the practice of traditional, ITIL-driven, IT service management falls too far over the line of control. So, that’s the habit to be wary of. Of course, the right cloud platform will balance standardization, automation, and developer flexibility.
Further Reading & Reading From You
At the outset of this series, I focused on greenfield and legacy journeys because there’s already a tremendous amount of work on transformed IT organizations. I find much of the transformation work out there thrilling and equally breathless—it all sounds great, but it leaves you wondering exactly what to do. There are some stand-out sources that cover actual tactics for transforming your organization in much more depth than I do here.
Still, there are two books I recommend strongly if you’re interested in transforming how your organization operates. First, Lean Enterprise is a concise, but comprehensive guide to all of the recent thinking about how to better run “normal” IT departments, much along the software defined business lines I’ve been discussing in this series. Second, for managers and as I’ve referenced before, Leading the Transformation is a good collection of tactics for managers to deploy—both up the chain to convince corporate overlords that transforming how IT is done is a good idea as well as down the chain to help staff in the IT department become a continuous learning organization.
Reaction from F100 CTO: “But Netflix has a superstar dev team, we don’t!”
@adrianco response: “We hired them from you.”
Finally, I’m curious to hear what you, dear reader, are finding out there. Simply knowing the problems companies are encountering as they’re trying to become Cloud-Native enterprises is helpful to everyone. Hearing about what has worked and hasn’t worked is helpful as well. To be overly recursive, that’s the broader mind-set we, in the industry and who want to improve the state of software, need to take—a continuous learning approach to the craft of the software life-cycle itself.
Part of that is sharing our experiences with each other and I’d love to hear yours.
- Read the first, second, or third article in the series.
- For a developer team level story of transformation, see the recent Cars.com write-up—revving up their agile approach to software.
- Understand the Pivotal point of view from the leader of our transformation practice.
- Find out more about Pivotal Cloud Foundry: Product Info | Blog Articles | Podcast
- Read our post-OSCON perspective on the The Cloud Native Journey
- Watch our CF Summit Keynote: Welcome to the Cloud Native Enterprise
- Get the Book: Migrating to Cloud Native Application Architectures
- Check out all the Pivotal blog posts, both technical and business oriented, on Cloud Native
About the Author