Congratulations! You just put your 100th application into the cloud, marking the end of a highly successful project. As the senior leader tasked with driving transformation in your company, you are celebrating with the team. Your project manager buys the first round of drinks—they can afford it because their bonus was directly tied to hitting the target of 100 apps in the cloud and they are happy to spread the cheer. Everyone is feeling good, not just about the project hitting its Key Performance Indicator (KPI), but also because you have managed to deliver on the vision of the company: You are embracing modern technology while transforming the way your business is run and you are ahead of, or at the very least keeping up with, your direct competitors. It is all win-win.
Later in the evening, after several more rounds of drinks and some of the more senior managers have drifted away, you start to chat with your finance team, who report to the CFO. They certainly don’t want to burst the bubble of the project celebration; however, they do mention that the run cost has gone up 200% over the duration of the project. You try to brush it off and say, “Well at least it’s all OPEX now, and since everything is based on subscriptions, costs will come down as people learn to use the cloud elastically,” but the finance folks do not look convinced.
Even later, and after more rounds of drinks, you meet the head of one of the larger app teams. They were a sponsor of the project; however, they also have concerns. “It has taken us just as long to get our latest deployment done even though our app is in the cloud. I thought we were going to improve time-to-market with this project? Also, I’m told by my developers that they are not actually spending any more time coding because we haven’t reduced the complexity of running an app now it is in the cloud—in fact if anything, it’s more complex.” “Well, technically, time-to-market and the increased number of hours spent coding weren’t the KPIs anyone asked us to measure in this project,” you reply defensively.
As you are settling the bar tab at the end of the night and only the stragglers remain, you overhear two developers laughing to each other. “This project was so easy. My KPI was to refactor five apps for cloud, but there was no way I had time to do that, and they didn’t really seem to care how I did it, just that I hit that number. So, do you know what I did? I conceptually split one app up in our inventory system, created five new entries and labelled them all as new “microservices.” Then I just lifted and shifted that monolith into the cloud and claimed I’d moved five apps. I didn’t even touch a line of code! All the project cared about was five entries in the inventory that said “moved to cloud” because it put their project status back to green on their KPI tracker. In fact, they were so impressed I’d done it so fast, they even sent other app teams to me to “help them modernize,” and because of that, I’m on the promotion list for this year!”
You are deflated—despite achieving everything the project asked for, you haven’t actually resolved the genuine issues affecting the company or achieved the outcomes you wanted. In fact, because of increased costs and complexity, things have become worse! Additionally, you’ve discovered that when basic KPIs are directly tied to people’s rewards and promotion, there is a risk they will “game the system” to appear as if they’ve achieved the results. In fact, not only did this project have no checks and balances for this behavior, it actively encouraged it because the “work arounds” achieved these surface-level, simplistic goals.
Lessons learned
So, what lessons can we learn from this cautionary tale of a transformation-gone-wrong?
We know we need KPIs as we need ways of tracking success. We also know KPIs drive behavior—especially when hitting that target is tied into personal rewards such as bonuses, promotion and recognition at senior levels. However, setting the wrong KPIs, as the example above shows, could have a detrimental effect on your overall business goals.
Why do we set bad KPIs?
Simplicity – Most people have heard of SMART goals (specific, measurable, achievable, realistic, timely), or a set of guidelines for setting personal objectives, however, a common mistake would be to apply the same logic to a KPI. Because in reality, a KPI is not a goal—it is simply an indicator of how close you are to hitting a specific metric. Our example (putting 100 applications into the cloud) is a very simple metric that can be easily tracked, and more importantly, is relatively easily and realistically achieved. So, it feels like a good KPI—even though on its own it has the danger of driving up costs and complexity.
Achievability – People are naturally inclined to set easily achievable targets. Most people are uncomfortable with complex or multifaceted targets whose ultimate success can be open to interpretation and based on someone else’s perspective. Similarly, people are less willing to commit to deliverables that are not within their control or outside the scope of the project. For example, if we changed the KPI from 100 applications into the cloud to decommissioning 100 applications on-premises, then the cloud project manager would correctly point out that they cannot be measured on this metric, because on-premises decommissions are controlled by a different department to cloud enablement.
Responsibility for KPI creation – As an example, allowing a project manager to set their own KPIs for their project in isolation means you run the risk that they will set a KPI they can achieve, rather than one that will benefit the overall goals of the company. Involving a wider group of people in KPI creation will increase the likelihood of achieving what everyone is striving for.
Time and effort – Time invested up front and by the right people, even if it appears more difficult or that the KPIs are more complex, is absolutely vital in ensuring the desired results are met, as well as avoiding the common mistakes highlighted in the story above.
What does a good KPI look like?
- Start with what you want to really achieve—For example, time-to-market for applications, reduction of an on-premises footprint (e.g., emptying an existing on-premises data center), reduction in overall cost (not just infrastructure cost), support for a reduction in risk appetite by ensuring vulnerabilities are addressed in hours, not weeks, etc. Then create KPIs that relate to these goals, rather than simply hitting a number. This could of course include scoping the project to cover meeting the real goals and not just the easily measurable facets.
- When selling to senior stakeholders, focus on the big rock achievements (e.g., cost saving, time-to-market, etc.) rather than relying on an impressive sounding statistic, such as the number of applications.
- Avoid picking arbitrary numbers. While “100” sounds nice, if modernizing and moving 87 applications manages to empty an on-premises colocation or a floor in a data center and that is your main goal, then set the KPI based on that. Similarly, if your developers are only really working on rapidly redeploying 38 applications while the rest are largely untouched for most of the year, then focus on those—as that is where you will get the most value and set your KPIs accordingly.
- Have multiple KPIs to avoid “gaming the numbers.” If, for example, you added a cost-measuring metric to your first KPI (of moving x apps to the cloud), then any gaming of the first KPI by moving inappropriate applications or double counting “real” apps would be exposed by not hitting the second KPI, as your costs would simply go up.
- Have someone external to the daily running of the project review KPIs regularly throughout the lifecycle. They should adjust the KPIs when necessary to steer the program back to the project’s original goals.
What do you want to achieve?
“I want to use the cloud to increase the lead time and frequency with which I can deploy my applications.”
Consider a KPI that measures lead time from code check-in to release to Production, as well as potentially another KPI that measures the number of releases over a period of time. This will ensure the project addresses issues such as app modernization and your Software Development Lifecycle process, not just deploying applications into a cloud.
“I want my developers to spend less time on getting infrastructure stood up and more time on coding.”
Consider a KPI that measures the decrease in lead time to spin up infrastructure, compared to on premises. If you already track how developers spend their time, you could consider a KPI that measures an increase in the number of hours spent coding.
“I want to move my applications to the cloud so I can exit a colocation, as the costs are far too high.”
Consider a KPI that measures decommissioning of applications and infrastructure in that colocation. Moving 100 applications to the cloud and leaving one behind doesn’t achieve your goal if you need to maintain that colocation just for one app. Equally you could consider a KPI that tracks the cost.
“I want to reduce my risk exposure.”
Consider a KPI that tracks how many vulnerabilities are addressed across the application estate and how quickly. Focus on the most important applications such as those that are internet-facing or deemed most business-critical.
What’s next?
Ultimately, it comes back to the goals and outcomes your business desires. A pilot project might want to move a few applications to the cloud just to prove the concept; so in this case, having the goal of simply getting those applications live in the cloud is perfectly fine. However, once the concept is proven and you want to move into mass rollout, then most businesses don’t want to just put 100 applications in the cloud just for the sake of it. They typically have other things they really want to achieve, such as time-to-market or overall cost. Remember that the cloud is simply an enabler, not a solution in itself, and should be seen as a means to an end, rather than the goal.
This article focuses on an example based on cloud adoption, however the same principle applies equally to KPIs in any situation. Therefore, for anyone planning digital transformation projects, it is vital to spend the effort at the start of the project and throughout its lifetime to set, measure, and adjust KPIs. Ultimately, you want KPIs to be working for you and towards your real, desired outcomes and business goals.
At VMware Tanzu labs, business outcomes act as the foundation for every digital transformation project. We bring stakeholders from IT and the line of business together to define a set of goals and align to real business outcomes, so we can prioritize and iterate on what works best.
About the Author
More Content by Darran Rice
