Being Agile Requires Strong Leadership. Just Not The Kind We're Used To

by Jeff Dalton

Everywhere we look, including every CMMI and Agile appraisal I’ve done in the last decade, technology leaders in large companies are asking about scaling agility. 

But it’s the wrong question. They should be asking how to scale self-organization. And in my new new book, Great Big Agile: An OS for Agile Leaders, I try to answer that question.

For centuries business has been led using a proven hierarchical, low-trust, command-and-control model that has its roots in the successful Roman military machine, and is still taught today in MBA programs from Cambridge to Ann Arbor. It’s a model that is in professional DNA, and self-organization, a foundational characteristic of agility, is absent.

In recent years, many businesses have been attempting to transition from traditional hierarchies to self-organizing models based on the  “rules of nature,” a system that more closely resembles the controlled chaos of the natural world. They start with the premise that humans naturally demonstrate certain behavior patterns, and it makes sense to leverage these, rather than re-program them to fit into more traditional hierarchical models. Agile frameworks like Scrum,  as well as self-organizing performance models like the Agile Performance Holarchy, are good examples of this. These models invert the hierarchy, transforming leaders into stewards of the a self-organizing behavioral architecture,  with team roles and accountabilities dispersed throughout the organization in a way that allows people go about the messy process of self-organization and improved performance. In an agile world, team members are empowered to make important decisions within the context of the behavioral architecture without having to ask permission from a supervisor or manager.

But don’t expect all mangers, or the schools where MBA students who are eager to lead are graduating, to come along willingly. To ask them to change is to ask them to transform themselves after a lifetime of learning how to succeed in a hierarchical world. This might have been best articulated by Benjamin Disreaeli, who upon becoming the Prime Minister of the UK, said, “I have climbed to the top of a greasy pole!” Indeed.

Why Agile Matters

According to the CMMI Institute, over seventy percent of organizations who have achieved a CMMI rating in the last three years describe at least some of their projects as “agile.” This is a dramatic increase over previous years that has deep-rooted cultural and operational implications. There are good reasons for leaders to transition to self-organizing models, but significant cultural change, especially among leaders, will need to take place to ensure success.

Agile frameworks reduce the cost of failure. It is conventional wisdom in the technology industry that failure is inevitable, with many companies seeing failure rates as high as 70 percent. Research conducted by organizations such as the Project Management Institute and the Software Engineering Institute has consistently confirmed high failure rates, so it makes sense to seek solutions that assume failure, not success, and to simply reduce its cost. All agile frameworks, with their incremental and iterative development model, support the idea of “fail-fast.”

Failure is not just an option; it’s a requirement. A foundational premise of agile is to acknowledge that failure is normal, and we should plan to fail fast and learn as much as we can. This reduces a project’s cost while allowing teams to redirect efforts toward a more successful approach through the use of experimentation, retrospectives, and short, timeboxed iterations. Quality professionals will recognize this as an application of W. Edwards Deming’s “plan-do-check-act” framework of continuous improvement applied in short iterations.

Agile methods deliver business value to end-users more quickly. Value is delivered more quickly with an iterative and incremental delivery approach due to low-value features being de-prioritized or discarded, freeing up valuable resources to focus on the high-priority needs of the customer.

Self-organization pushes decision-making downward, freeing leaders to focus on strategy. For decades, the technology industry has explored ways to push decisions downward. Agile frameworks finally provide a model that can make that a reality, if only leaders are willing to accept their role as enablers rather than task managers. A successful agile team requires minimal over- sight, makes day-to-day operational decisions, collaborates with business customers, and delivers business value without the need for continuous management intervention.

Agile complements important IT industry models. If CMMI®, ISO 9001, and the PMBOK® Guide are models we use, agile is something we are. For example, CMMI has a perspective of defining what needs to occur for a product or service to be successfully and consistently deliver, and to improve the process,  while agile values describe why we take those actions. If adopted in this way, rather as only a marketing tool to receive a rating, CMMI makes agile stronger.

Big Agile is Coming

Since 2016, General Motors, the Department of Defense, Health and Human Services, Fiat Chrysler, and other large companies have begun to adopt agile within their software organizations, and along with their combined $100 Billion IT budgets they are bringing their biases, bureaucracies, documentation, and leadership infrastructure with them. What will be the effect on the agile community?

“Big Agile” requires leadership at all levels, just not the kind we are used to. Simply working with an agile coach to implement well-known ceremonies is not enough. Metaphorically, the leadership “operating system” needs an upgrade.

In today’s corporate hierarchies where command-and-control structures, low trust, long-term planning, and risk management reign supreme, the skills required to thrive and survive are anything but agile. This leaves agile teams to push the culture uphill, leading to unpredictable results once business operations expand beyond the boundaries of the core agile team. This creates a “cultural type-mismatch”  due to information technology, operations, marketing, infrastructure, business development, sales, and end-users not being on the same cultural page.

Performing agile ceremonies and techniques without self-organization isn’t agile at all. There is nothing inherently wrong with adopting ceremonies and techniques identified as being agile, and many companies have found some success with that, but the power of agile values and their associated frameworks grows exponentially once self-organization is perfected.

What To Do About It?

Tomorrow’s leaders, and the schools and corporate mentoring programs that train them, will need to transition their mission from that of command-and-control task manager to one of an architect and operator of a self-organizing infrastructure. This includes changes in culture, training, and performance monitoring with a bias towards high-trust, peer accountability and self-governance. One model that can help new leaders prepare for the future is the Agile Performance Holarchy (APH) from AgileCxO.org.

The Agile Performance Holarchy is a leadership model that provides a definition of a self-organizing agile architecture, with objectives, desired outcomes, and set of behavioral guiderails for agile leaders and teams seeking to master self-organization and large-scale agility. The APH currently contains six Performance Circles that address Leadership, Craftsmanship, Providing Infrastructure, Affirming Quality, Teaming, and Envisioning Solutions.

Culture Needs To Change

An “Agile Transformation” where the scope is hiring an agile coach, and the adoption of basic ceremonies or techniques, is doomed to failure. Agile isn’t a process or a framework like scrum, XP, or SAFe.  It’s a collaborative, transparent, and self-organizing culture where the operational model is high-trust, empirical, and calibrated for relentless improvement in the pursuit of high quality and increased speed to value. 

Jim Bouchard, author of The Sensei Leader, sums it up for leaders: “Don’t even attempt to transform your organization until you can transform yourself.”

About the Author

Jeff Dalton thinks the future of Big Agile is our industry’s biggest challenge, and he has been studying it for years. As the large adopters in the federal government and corporate sector begin to adopt agile, they will bring their habits, culture, and bureaucracies with them, and in dozens of podcasts, articles, books, and keynote speeches Jeff has been talking about getting in front of the wave. A veteran technologist and IT leader, Jeff started as a software developer and has been a CEO, chief technology executive, vice president of product development, director of quality, and Agile evangelist for over 30 years, including time with Ernst and Young, Electronic Data Systems, Hewlett Packard, Intellicorp, Polk, Broadsword, and AgileCxO. As a consultant, teacher, CMMI lead appraiser, and leadership coach, he has worked with NASA, Boeing, Accenture, Bose, L3 Communications, Fiat Chrysler Automotive, General Motors, Ford, and various federal and state agencies to help them improve performance. Jeff is a frequent keynote speaker, Agile performance holarchy assessor, blogger, and host of The Agile Leadership Podcast, a monthly series that interviews CIOs from state government about the challenges of agile adoption. In his spare time he is an instrument-rated pilot and plays bass in a jazz band.

This article was contributed by Jeff Dalton, author of Great Big Agile.