Agile Ecosystems and the Prisoner’s Dilemma

Added to People

There's another "systems" related topic I want to touch on before moving on with Agile Tribes. It will tie together some factors that can hinder Agile adoption in the small (within one business) and in the large (industry wide), and the "tribal" nature of the community.

One question that can be legitimately asked about Agile (and indeed about anything that is proposed as a Better Way of Doing Things) is as follows:

If this is such a hot idea, then organizations using it should outperform others. Organizations are subject to a sort of economic Darwinism - the better performing ones stick around, the others die. Over time we should be seeing more organizations using this Better Way. Why hasn't that happened yet, with Agile over ten years old?

Some possible answers are: a) actually, we do see this happening, or b) ten years isn't such a long time given the lifespan of organizations. If you have reason to believe that a) and b) are ruled out, then we seem to be left with c) it wasn't such a hot idea after all; looks good but doesn't work in the real world, back to the drawing board.

Enter the Prisoner's Dilemma, a staple of Game Theory. This is a two-player game with two possible moves, "Cooperate" or "Compete". The game is about winning as many points for yourself as you can. Suppose the two of us are playing - if you choose Cooperate and I choose Compete, I win five points - and you win nothing ! Frustrating ? But on the other hand, if you choose to Compete while I Cooperate, you win the five points and I get nothing. If we both Cooperate, each of us wins three points. Finally, if we both Compete, each of us wins a measly one point.

The names of the moves are merely suggestive - they have nothing to do, necessarily, with how best to play the game, and do not imply a moral judgment. What matters is the outcomes of the game, summarized in the "payoff matrix", as below:

The Prisoner's Dilemma yields a good model, though hugely simplified, of a situation in which good ideas fail. Imagine, if you will, an economy based on playing that game. Imagine further that everyone - all the businesses in that economy - typically play the Compete move. With any interaction, any two businesses make some little profit - one point.

In that context, Cooperation is demonstrably a Better Way: it would be possible to get three points rather than one out of every interaction if everyone was using it. Why doesn't everyone start doing it ? Because it makes no sense for any one business to switch their strategy to Cooperation, unilaterally. They would be "rewarded" by going up against Competitors and gaining no points at all from any business they did. It would take a coordinated decision for everyone to move to Cooperation, but in a "free" economy there's no opportunity for such coordination.

This seemingly desperate situation is called a Nash equilibrium, after John Nash, the mathematician who studied them.

Robert Axelrod's book, The Evolution of Cooperation, tells the fascinating story of a series of experiment involving games of Prisoner's Dilemma conducted as computer simulations and "tournaments". I strongly recommend the book as your first stop if you're interested in learning more about the game, or about game theory. It contains little that looks like "theory" - equations or long logical arguments - but mostly focuses on the results of simulated games, pitting against each other strategies submitted by some of the best mathematical and scientific minds in Axelrod's distinguished entourage. The results have practical relevance to anybody interested in how Better Ways spread... or don't.

In particular, Axelrod shows how Cooperation can in fact - almost against all hope - win over Competition: our models (and Axelrod's simulations) need to introduce the notion of territoriality - i.e. the idea that anyone's interactions in the game will mostly be with "neighbours". (They're not necessarily neighbours in physical space - "idea space" could be a good way to think of it.) In Axelrod's simulations, it turns out that small clusters of Cooperating neighbours can eventually "invade" a population stuck at a Nash equilibrium. (It turns out that "always Cooperate" is not a long-term viable strategy, but an extremely simple strategy called Tit for Tat is: it consists of choosing Cooperate, except if your opponent choose to Compete in the previous round, in which case you "punish" them by Competing once and then, "forgiving" them, go back to Cooperation.)

Many issues in Agile, from small to large, can be modeled with the Prisoner's Dilemma or some extension of it. (This has been discussed from time to time in the Agile community; I wrote an earlier version of this article in 2005, but probably wasn't the first to make this connection.)

For instance, it may not be in a developer's best interests to unilaterally start writing automated unit tests for the code they write, if no one else on the team is doing that and developers are frequently switching from one area of the code to another: that developer will be seen as "less productive", since she spends some of her time writing tests rather than code; but she won't really benefit from the quality increase, since others will soon be changing her code in ways likely to degrade its quality. Yet (assuming that unit tests are in fact beneficial) everybody would be better off if they did write these tests: the team as a whole would be seen as much more productive. This may also apply at larger levels, for instance several teams that are part of one large project, where each team is tempted to create technical debt to "go faster".

Another area of application is Agile contracting. The most prevalent mode of contracting at the moment is largely based on mutual mistrust, and the incentives are set up to discourage either "player" from unilaterally choosing the "Cooperate" move: from the perspective of a software vendor, agreeing to the Agile approach of allowing changes throughout the project will not improve their bottom line, and they may be blamed for not delivering the agreed upon scope; from the perspective of a client organization, an Agile process means having no fixed limit on how much a project will cost. Both would benefit from mutual adoption of Agile, but one-sided adoption is not a "rational" move.

Another area is individual skills development. At the Agile India 2013 conference Dave Hoover observed in his talk "Create Your Own Apprenticeship Program" that organizations tend to play a "Compete" move when it comes to growing people's skills: they act solely as "consumers" of technical skill and never as "producers", failing to allow the requisite time and resources for developers, testers and other specialties to invest in ongoing learning. In that context, unilaterally making the move to invest in excellence may not seem like a rational move: what's likely to happen is that your best people will be "poached" by other organizations that do not themselves invest in learning.

These are systemic issues that go beyond the mere formulation and refinement of Agile thinking, practices and advice. No amount of explaining Agile better or improving our ideas  is guaranteed to make this problem go away.

What Axelrod's simulations suggest is a key function for "tribal" gatherings and networks. They ensure that "players" in the game who are likely to choose Cooperation can recognize each other, and preferentially choose to do business with each other.

Dave Hoover's example is enlightening here: his success at creating apprenticeship programs took place in a particular context, that of a small set of companies in Chicago committed to "software craftsmanship", such as Obtiva or 8th Light. Knowing that similar companies were also investing in skills development would have made it a sounder decision for any one of them to do so as well, limiting the "poaching" problem. (Obtiva was eventually "poached" wholesale, acquired by Groupon - I don't know what effect this had on the rest of the Chicago scene.)

Similarly, the Agile Executive Forum that has been co-located with the Agile conference in the past two years will possibly serve the function of drawing together a "network" of organizations that can benefit from setting up mutually beneficial commercial dealings, without running the risks inherent in unilateral Cooperation. (As a counter-point, there is a growing concern that some organizations which are not "true" Cooperators may try to signal allegiance to Agile without actual changes in behavior, for instance by sponsoring Agile events. Here again, personal relationships are key to identifying such "wolves in sheep's clothing".)

The future of Agile, thus, may well depend on a better understanding of how tribal networks form and develop.

 

About the Author

No bio currently available.


This is an Agile Alliance community blog post. Opinions represented are personal and belong solely to the author. They do not represent opinion or policy of Agile Alliance.