Design and Agile’s Muddied Waters

What seems a pedantic debate to some will for others mark a deep and troubling conflict over the nature of their work. This has long been the case for Design and Agile. To take current example, designer Pavel Samsonov recently argued that the notion of “UX within Agile” cannot exist precisely because Agile really isn’t a thing.

Many would laugh at this as more pedantics, as the ramblings of yet another “Agile Is Dead” post. Only not quite. Pavel is quite correct in his assertion that if Agile largely wants to define itself as an alternative to Waterfall, then, yes, it’s not really a thing. Or, rather, it then exists largely as a false dichotomy. Making matters worse, other than pointing to a now decades-old Manifesto, no one can seem to agree what the hell “Agile” is anyway.

As we’ll see in a coming post, even the Manifesto signatories don’t really seem to agree. To quote Spencer Pitman, Agile is so vague a notion that it’s hard to even have a meaningful debate about it. This is reminiscent of Charles Betz asking whether Agile might possibly be an “essentially contested concept”, a term the proper use of which seemingly cannot be resolved by continued discussion or debate.

Well, if it can’t be defined, then what’s the use discussing it? The answer is because we’re forced to, of course. Because it’s a debate that continues to wreak havoc on the way we work. In this post I would like to suggest that the Agile debate has largely served to obfuscate how we work and that, further, the contrasting of Agile and Waterfall needs to stop.

An Unhelpful Dichotomy

Agile and Waterfall practically serve as character foils, each largely now understood vis-à-vis the other, and with no small amount of revisionist history. As noted above, it is ultimately a false dichotomy. There is always a “waterfall”. You can increase agility within it, but to paraphrase Samsonov, Agile as a wholesale alternative not only rarely exists but might not even be a good idea. As Alfredo Zangara recently put it to me, there is much about what is now considered “Waterfall” that you should not want to lose.

Consider that at the product level one will rarely find a single, cross-functional team responsible for everything. It is more likely a great many teams involving hundreds of people. The product will have a life cycle, as all do, and there will be handoffs between the teams. Furthermore, the product must first come from somewhere, which means there should be initial work that is very different in nature from later work.

As Don Norman notes in The Design of Everyday Things, with almost any product there will be phases it will be difficult to return to as more decisions are made—which brings one closer to “Waterfall”—and there should also be an iterative refinement of assumptions as progress unfolds—which brings one closer to “Agile”. (But why even call it Agile? Design knew this long before Agile was even a thing.) Try as you might then what you’ll end up with will be some blend of the two…and Norman argues this is as it should be. As he puts it, the goal should be to have the “best of both worlds”.

Johanna Rothman, in Diving for Hidden Treasures, similarly argues that the aim is not to “become Agile” but to maximize value, which will often entail slicing work as small as possible, focusing on good acceptance criteria, and leveraging frequent milestone reviews to reassess risk and regularly reprioritize investments.

So, the question arises…is this Waterfall? Maybe. Is it Agile? Who cares?

The important question is: Why does it matter?

Where the Hell Is Strategy?

In most organizations, Scrum has filled the gap left by the vagueness that is Agile. This leaves us with a reality where “really existing Agile” is little more than Scrum imposed in a top-down fashion on dev teams. Allen Holub calls this “Scrum at the bottom”. As Dan Sloan notes, at this level there is typically little to no responsibility for strategy. This confines iteration to tactics and does not help the larger organization itself become nimbler. In other words, it sets a low “agility ceiling”. Teams, after all, cannot iterate past the limits of their own decision authority.

Scrum then leaves us with a “really existing product work” where there is no actual product vision or strategy in sight. If your dev teams interface with users and POs prioritize what to build, that’s not a product strategy—that’s just process. This abdication of strategy is, I would argue, a bigger issue than whether teams are “Agile” or “Waterfall”. As Patricia Colley stresses, a real product strategy—and the work that would go into creating a good one—is vital to providing product with conceptual integrity.

The solution, which SAFe at least tried to do, is to explode the confining of Agile to dev teams. The title of the Manifesto, designating Agile to the realm of “software development”, should be altogether rejected. This diminution distracts from the larger task of picking apart the tight relationship between real agility and organizational decision authority, thereby responsibly raising the agility ceiling in the organization itself.

Scrum does not help with this. You frequently share your work and get feedback. You build what others say they need. Proceeding thus, the larger architectural work—in its proper design sense—will be missing. It’s tactics all the way down. Taking a page from Jabe Bloom, this ignores time horizons. It ignores a sound product vision, which grounds the strategic intent. It ignores building out a nested, causal model, where the difference between tactics and strategy lies in dialing up from short-term present HOWS to longer-term future WHYS.

Agility as Optioning

Other frameworks can be leveraged to help raise the agility ceiling without sacrificing strategy, such as VMOST analysis (Vision, Mission, Objectives, Strategy, and Tactics) and/or the North Star Framework. Consider the image below, adapted from Rob England and Cherry Vu’s excellent book, S&T Happens. At the top, conventional practice has short, medium, and long-term planning that does not account for VUCA (volatility, uncertainty, complexity, and ambiguity). Annual plan, for example, dictates the year to come as though a crystal ball was at hand.

The more VUCA the world, England and Vu observe, the more this drives a wedge between short-term planning for the foreseeable future and long-term visioning for the far future. Any traditional planning within this “zone of uncertainty” is therefore risky (and likely waste). Thus, the traditional project with its output-based roadmap and milestones will likely be high risk. Moving down, the more you acknowledge VUCA the more you should approximate some form of scenario planning (such as assumption-based planning).

Here, one keeps real options alive in the form of possible futures. The focus then shifts to these options and the operational agility required to act on them at the last responsible moment, as indicated by changing conditions. At the bottom of the diagram, this is approximated by translating options or scenarios into alterative possible objectives and their respective provisional outcome-based roadmaps.

Don’t Scrum Your UXD

We’ve talked about “Scrum at the bottom”. Well, same goes with User Experience Design (UXD). UXD, at its core, is not about making user interfaces look pretty. As Erika Hall defines it, design is the orchestration of the exchange of value within constraints to achieve a goal. In other words, design is about discovering the best problems to solve and exploring what you should try out as you solve them.

The real medium of any design then is decisions. The more decisions are already made, the more the direction is set, and the fewer “degrees of freedom” remain. As the degrees of decision freedom are “spent”, there is less wiggle room for agility. The opposite of Agile isn’t “slow”—it’s “path dependent”. To raise the agility ceiling in the organization you need to bake options into planning to preserve degrees of freedom.

The discovery of these options and the feeding of this scenario-style planning approach will largely be driven by the work of UXDs. As Jonathan Korman argues, though Product Management (PM) should have final say on product strategy, thereby ensuring it feeds into higher-level business strategy, it should nevertheless be primarily derived from the work of UXDs—and this cannot happen if you confine UXD work to the dev-team level.

The larger the initiative, the more PM will reside above the level of dev teams. To raise the agility ceiling, then, UXD must be similarly raised. And yes, this higher-level UXD work will largely take the form of research. There is a bizarre irony in rejecting such upfront research as being somehow anti-Agile “BDUF” (Big Design Up Front). Scenario planning, after all, both requires a lot of upfront research and is precisely the sort of thing that helps ensure organizational agility in a high-VUCA context.

Also ironic is all the usual obsessing over “velocity” when teams are deprived of the conceptual integrity of a good vision and strategy. (In fact, that probably feels a lot like gaslighting.) As Korman has put it, so much of this obsessing is itself a result of an underlying lack of strategic clarity. As he notes, if you have a clear strategic vision of system intent, then making tactical decisions becomes simpler and “done” becomes easier to talk about.

To close, there are of course multiple ways to build this out. An interesting alternative I learned about from Rob England is the Last Planner System^®, from the world of construction. As shown below, this maps quite nicely to the structure played with above.

Until next time.

If you’re interested in coaching, contact me.

You can also visit my website for more information.