Scaling Leadership of an Agile Software Factory: Lessons Learned

June 10, 2020 Jason Fraser

This post was co-authored by Jason Fraser and Jackie Ho.

About three and half years ago, VMware Pivotal Labs embarked on a journey to bring an organic agile software development capability to the “world’s largest bureaucracy” (the U.S. federal government). There are countless vendors that could have provided agile development for this group. But our client wanted to learn to do it themselves. Our client wanted a “factory,” with the ability to churn out valuable software outcomes for its internal clients.

When building an agile software factory, there’s often a strong desire to scale quickly. There’s a sense that larger efforts contribute more value and are easier to protect than smaller efforts, and the desire to “get more done” can lead to the conclusion that “We need more people!”

There are some dangers to scaling your lab effort too quickly, though. The most significant is that if you scale your product teams without first scaling the leadership team that will support them, your effort can suffer from employee attrition, failed onboarding of new hires, and time wasted on building products that are misaligned with your goals. We saw all of these side effects and more, and this article will discuss our lessons learned from scaling the leadership team for our client.

In just 2.5 years, this client’s program grew from 1 team to 18 and from 5 people to 250. Along the way we learned:

  • Three reasons that leadership needs to scale in advance of the rest of the organization
  • Three signs that it’s time to scale a leadership team
  • Three signs that we’ve done it right
  • How to identify the right positions to fill
  • How to find the right people

Our work and our learnings are primarily centered around the level(s) of leadership between a product team and a director. For the first two years of this organization (which you can think of as a line of business), there was a single director and 18 product teams with little to no support structure between them (no managers, no associate directors, no portfolio or program leadership).

Three reasons that leadership needs to scale in advance of the rest of the organization

The client grew its product organization faster than it was able to grow its leadership team. As it did so, we made three key observations.

1. Product teams were neglected

Insufficient leadership capacity during scaling resulted in new teams being spun up that subsequently had very little direction from leadership. These teams suffered from a lack of alignment with the organization from the outset, and in some cases wasted effort on finding a path forward through complicated issues that could have been easily managed by someone in a leadership position

Additionally, existing teams suffered neglect on an ongoing basis. As the organization grew, leaders became less and less accessible to their teams. Teams were blocked on matters that only leadership could resolve, staffing changes were poorly communicated and painful, and teams lacked product direction and alignment with the rest of the org. Some teams drifted and stalled out (with costly implications). Significant time was spent on things that had little value. An investment in leadership could have saved more than it cost.

2. There is a lag time to productivity for new leaders

It takes time to onboard new leadership and get them to a place where they’re decreasing the workload on others rather than increasing it. Our new leaders were thrown into a maelstrom, and it took some time before they had any positive impact on the organization. Ramping up those leaders in advance of demand would have given them an easier transition with a lighter workload so that they could have spent more time understanding the work before having to do it. This team was also defining their leadership roles and responsibilities in real time, adding a layer of complexity to the transition.

3. Ramping up people and teams requires leadership support

In order to support the hiring and onboarding of new people and the establishment of new teams, you need leadership capacity. New people were showing up every day and in many cases, no one even knew they were coming. There was no one to greet them, no one to help them through their own onboarding process, and no one to tell them where to go or what to do. This caused anxiety among new hires, and made everyone in the organization feel a bit guilty.

Three signs that it’s time to scale a leadership team

We found three clear signs that it’s time to scale your leadership team (or that you’re already behind).

1. Leaders have zero slack time

When leaders have wall-to-wall meetings in their calendars, thinking and action tend to shift from proactive to reactive. Leaders feel overwhelmed and ineffective. Everything feels like a fire drill, and there’s never time to stop and develop strategies or build systems to eliminate problems rather than treat symptoms. This lack of strategic thinking was felt acutely by product teams and leadership alike. 

2. Leadership is disconnected

When the leadership team is overwhelmed, its members will start to become disconnected from what’s happening with teams, products, and people. A lack of touchpoints and support can lead to discontent among the staff, and unwelcome surprises during check-ins. No one likes negative surprises.

3. Leadership lacks necessary skills

You may also get to a place where your leadership team doesn’t have the full set of knowledge or skills necessary to move their teams forward. That skills gap can become apparent to the team and can lower trust in leadership. 

Early leaders in fast-growing organizations are often chosen for their availability rather than their skills. As the organization grows, it pays to be pickier about who moves into leadership positions, choosing for specific skills and aptitudes rather than simply choosing the most senior person who’s willing to step up. For example, early leaders in this case lacked technical expertise, and the organization later benefited from adding members to the team who would  own any platform-related issues. 

Three signs we’ve done it right

When organizations do this right, they see three clear signs.

1. Positive health metrics and survey results

If you do pulse surveys (not everyone does), you’ll start to see higher ratings around things like organizational efficacy and employee support. The client we worked with used health metrics with its teams, and one of the things it measured regularly was leadership support. As the leadership org scaled and brought in people with the right skills, those ratings went up.

Team running a health check gets honest about their challenges with leadership support.

2. Abundant and effective collaboration

When you have enough people on the leadership team, they have the time to collaborate with each other, with their teams, and with individuals. This collaboration can drive alignment in addition to esprit de corps. Things like continuous improvement, long-term planning, exploring growth opportunities for people, and making deliberate decisions about which efforts to pursue next suddenly become possible when you’re not fighting fires all the time due to limited capacity.

3. Deep connections with teams and individuals 

Leadership feels more connected to the people and teams they support. They know what’s happening. There are no surprises during check-ins. Issues are resolved in a more timely manner.

How to identify the right positions to fill

Identify your weak spots

When bringing on new leadership, first identify your weak spots. Then you can select for people who are able to contribute meaningfully in areas where you’re weak. It’s OK to change leadership structure or add new roles as you’re growing, too, as long as you do it thoughtfully and communicate well.

Explore existing patterns

Existing leadership patterns or organizational structures can help you figure out the best way forward for your organization. Don’t assume that one company’s model (<cough> Spotify) will work for you out of the box. Most organizational structures are a combination of evolution and engineering. For example, this client’s leadership team evolved around its staff’s relatively low experience level and a critical need for collaboration and alignment across product teams. Nature and nurture both play complex roles, and while your organization’s nature and the way it nurtures might be similar to another company, you’re likely to be unique enough to require some customization.

Try some experiments

Experimenting with structures and roles is great if you have the freedom to do it. Can you get someone to take on a different role for a little while and see if the role works? 

As with any experiment, it’s not an experiment if you just “try it and see what happens.” A real experiment takes discipline and rigor. Define your hypothesis, decide on your time limits, and log your results. What do you expect to see? How long would you have to give it before you might see results?

Sharing the results of your experiments with the rest of the organization can help others learn and make better decisions, too.

How to find the right people

Promoting internally is almost always better

A good culture can be damaged by a misaligned leader. Leaders who come up from the team ranks are usually more aligned with the mission and culture of an organization and may take less time to ramp up.

In some organizations (like this client), domain context is hugely important. The workflows that are being instantiated in software for this client have been 70 years in the making (with a major realignment about 30 years ago), and knowing what’s what takes a lot of effort and experience. Without the domain knowledge, though, decision-making is impaired.

Hiring from outside your organization can also limit growth opportunities for people who have been loyal from the start. And if not managed well, it can engender pessimism. If every time a new leadership role is opened up, it goes to an outsider, people can feel like their contributions are not valued, or their growth paths are limited.

Outsiders are sometimes the best choice

That said, outsiders can bring badly needed domain knowledge and/or practical skills (engineering/PM/design/etc). Sometimes an outsider who comes from your user community will bring more domain knowledge than the people already on your team. If they’ve got management skills as well, they can be extremely valuable. The same is true for practical skills. Salting the team with some highly skilled leaders (who have the time and space to share their knowledge) can raise the skill level of the whole team.

Finally, look for opportunities to hire for culture add. Consider what a person could bring to your culture that isn’t already there or that exists and you’re looking to promote further. If you’re bringing in someone from the outside, there’s an opportunity to add a fresh perspective while maintaining the importance of empathy, curiosity, respect, and humility. Watch out for surface-level signals or appearances that may lead you to think someone has the same values as you, however; if it’s not obvious, ask. Taking time to understand a potential new hire and the ensuing cultural changes that person would bring to your organization will prevent potentially irreparable damage.

Conclusion

Scaling a leadership team is hard and often painful. There are many moving pieces to consider, and it can feel impossible to make everyone happy. Through all the changes and ups and downs, the best leadership teams prioritize good communication, one-on-one feedback from practitioners, and frequent and honest team retros. With these practices and a desire to learn and improve, leadership teams can grow and evolve together effectively.

Previous
The 1-Factor App: Using Kubernetes to Get a Jump Start on Modernization
The 1-Factor App: Using Kubernetes to Get a Jump Start on Modernization

Full app modernization is hard, but being able to restart is a good start.

Next
Process, Automation, and Continuous Everything: Analyst Report Insights
Process, Automation, and Continuous Everything: Analyst Report Insights

Dormain, Rita, and Derrick dissect three analyst reports on modern infrastructure and operations trends, ap...

Gartner Report on Application Modernization

Get the report