Why Agile and UX Still Don’t Get Along

Agile is an odd beast. It’s not a field, like UX. It’s not a framework, like SAFe. It’s more a mindset centered around a manifesto. The Agile Manifesto, though, is just that—a manifesto. It’s brief. What it doesn’t say is voluminous. What it does say is often ambiguous. Agile’s legacy is difficult, to say the least. Its overarching narrative has devolved into a sort of religious war, with Agilist Protestants acting like the manifesto was Luther’s Disputation of the Waterfall Catholic Church.

And yet, despite two decades of hyperbole, there is a sense in which Waterfall isn’t going anywhere. As Norman argues in the newest edition of The Design of Everyday Things, large projects still benefit from decision gates and iteration still typically makes the most sense between them. Indeed, it might be time to admit the whole “Agile vs. Waterfall” thing was a red herring and a waste of time.

After all, the real reason most organizations want “Agile” in the first place is not to move away from Waterfall but to speed it up. It’s an efficiency gain, an attempt to “squeeze more water from the Waterfall”, as Grigori Milov has put it. Unfortunately, this is often approached in an odd, cargo-cultish way, with organizations adding roles, rituals, and even complexity in an ironic attempt to “do Agile”. As a result, such transformations inevitably fall prey to what Gerald Weinberg called “Prescott’s Pickle Principle”.

Of course becoming more efficient is important, but so is improving product strategy, and this is where Agile continues to be inhospitable to UX. Agile, at its core, is supposed to be about the ability to pivot. This should raise a key question: Where should these pivots come from? How are we to know when to pivot and to what? Borrowing a phrase from Tom Kerwin, what are the best “pivot triggers”?

This, and this precisely, is where Agile and UX fundamentally disagree. Now some might claim this is only true of “Dark Scrum” or “Fake Agile”. I’m increasingly convinced this isn’t the case. There is no end of debate about Agile, but the one thing people seem agreed on is the value of the manifesto, and, well, there is actually quite a bit in the manifesto that conflicts with UX.

For instance, the first principle states, “Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.” From a UX perspective this is already suspect. Both “valuable” and “customer” here are unhelpfully vague, and it turns out that by “customer” the manifesto coauthors largely meant “business stakeholders”. “Valuable”, then, is whatever business stakeholders say will be of value.

The manifesto then states that developers should work daily with the business and welcome surfaced “changing requirements” as customers (stakeholders) sample the “working software” provided. Agile’s value proposition then lies in allowing said stakeholders to sample working bits of their upfront guesses and then, if so desired, to ask for something else (i.e., to “pivot”). This avoids the trap of placing all eggs in one basket (or launch) by breaking larger bets into smaller ones.

Sounds good, right? So…where is the disconnect? Well, from a UX perspective, this treatment of value is circular; further, filling stakeholder solution requests—even in an Agile way—does not break this recursive loop. As a result, this leaves design and its core function completely out of the picture. It sidelines the role of discovery and treats teams like line cooks in a fast-food restaurant. This overlooks the actual end-users, their goals and frustrations, and, in so doing, ironically ignores the only thing that actually does create business value, which (spoiler alert) isn’t software.

As readers of this blog (hopefully) well know, the only way to increase value is to change someone’s behavior. Whomever that someone is, whether user, customer, employee, or constituent (it really doesn’t matter), there is no value created without the behaviors that actually create and sustain it. For UX, then, the only form of agility that matters is the ability to alter the behavioral outcome being produced, and this does not hinge on what business stakeholders happen to request as much as the manifesto seems to assume.

It hinges, rather, on the ultimate experience provided and the team’s ability to move from symptomatic issues to underlying needs. And this raises a further question. Should teams mainly focus on building software…or on meeting needs and solving problems? These are not the same thing. From a UX perspective, rethinking an employee’s day, redesigning a workflow, or pushing for policy changes nixing the need for some tedium are just as viable as quickly coding the next feature. Ignoring this leads us to Weinberg’s “Law of the Hammer”, a point also famously made by Abraham Maslow:

UX avoids this by insisting we be more consultative than Agile seems to think teams should be. UX’s approach is more like medicine, where taking a patient’s self-diagnosis at face value would be considered malpractice. Breaking large bets into smaller ones is all well and good, but UX is more about making smarter bets; and, at its core, takes umbrage with the notion that stakeholder requests are the best signal of unmet needs (or even a good one at all). The bottom line is that what people know how to ask for is often not all that predictive of future behavior, and behavior, remember, is the lever at play.

Indeed, as Donald Norman frequently cautions, what stakeholders think is the problem is often not the real problem. Pivots then should not be driven chiefly by changes in stakeholder requests as much as by contextual research, which should not be shortchanged for efficiency gains to place poor bets faster. Now, when I make such points many reply that Agile teams should be doing user research and that there is no real conflict here. This is easy to say, especially while glossing over the inconvenient fact that Agile and UX often mean very different things by “research”.

The real medium of design, after all, is decisions made, and the main barrier to increased value is decisions already made. You cannot iterate your way out of a problem frame dictated by decisions you cannot unmake. Ignoring this robs product teams of much of the value they could otherwise create, minimizing their impact by confining efforts to a small, tactical box. A product team doing their own research at this low level and quickly shipping coded product will not even be focused on the same kinds of problem-framing issues that upstream contextual research will unearth.

As Pavel Samsonov notes, this highlights the foolishness of organizations attempting to fit UX “inside” Agile. UX, properly applied, must be leveraged at the level such decisions are in fact made, and this is often far above the product teams. And yes, agility too should ideally be driven upstream, to the level of planning and finance. Unfortunately, that is not what the manifesto is about—it is specifically about building software. (Additionally, most organizations are as uninterested in actual agility as they are in leveraging research to derisk assumptions.)

To close, Agile’s issue with UX can be boiled down to the common assertion that Agile is somehow “bigger than” or “above” user experience, which is an odd take. At the end of the day, Agile desperately needs something to fill the product strategy gap it left in its wake, and not because it failed at product strategy as much as it assumed it didn’t need to have one in the first place.

That something is proper UX Research, which organizations, to their own detriment, increasingly seem to think they can do without.

Until next time.

If you’re interested in coaching, contact me.

You can also visit my website for more information.