A structural defect, not a personal failing
Technical debt is the deficit in code quality that adds up over time. When you copy and paste code, create classes with unnecessary complexity, or commit other coding “sins,” you make it harder in the future for you or other developers to work with the code.[1]
Technical debt is a fact of life in software development. Even if no one touches the code, the technical debt will increase marginally. Most problems with technical debt occur, however, when developers do a less-than-disciplined job writing code. That’s when technical debt reaches a level where occasional refactoring cannot deal with the problem.[2]
At first glance, technical debt looks like a problem that is easy to solve, and avoid in the first place. Lack of discipline in writing code translates into later problems expanding or fixing the code. Productivity suffers in conspicuous ways. If they are willing to make the commitment, software developers can clean up the technical debt they have created. With added discipline, they can avoid accruing debt all over again.
In reality, reducing and avoiding debt is not nearly that simple. Technical debt is a ubiquitous condition of software development, not a problem for a minority of undisciplined developers. Except for minor exceptions, the question is not whether there is technical debt, but instead, how much there is, and how much impact it has on individuals, the team, and the organization.
A bad relationship between the individual and the organization
Why do teams accumulate technical debt, even when they know they shouldn’t, and they feel its drag on their work? When I run sessions of Dice Of Debt, [3] a game about technical debt prevention strategies, I ask the participants after the game, “If you’re willing to invest in technical debt prevention in the game, why don’t you in real life?” The answer is always a variation on the same theme: “It’s complicated.” The people making this statement, dedicated software professionals, who are looking for ways to deal with their technical debt problems, are using the same language (and often the same intonation) as they would use to describe a troubled relationship, such as a marriage or friendship.
That’s no accident. The roots of technical debt lie in the troubled relationship between the developer and someone else – not a spouse, in this case, but the larger organization in which they work. While individual developers may make mistakes that contribute to technical debt, the problem isn’t merely the junior programmer who has not yet learned the value of good code review, or the cost of copying and pasting code. In fact, individuals often bear unfair blame for technical debt, as expressed in statements like, “If only we had better developers, they could produce more, and do it more reliably.” Sometimes, the real problem is not the developers, or the teams of which they are part, but the system in which they operate.
Complex systems create complicated relationships
Technical debt is not merely a byproduct of how these systems operate. Technical debt also changes teams and organizations. Tragically, these changes can magnify the pressures that lead to technical debt, creating a nasty positive feedback loop. For instance, a team under severe time pressures may cut corners writing the code to meet a deadline. The technical debt that results makes it harder to write future code, creating another potential deadline crisis.
Technical debt therefore shows all the traits of a systemic problem. No one wants a bad outcome – in this case, accumulating technical debt – but the way the system operates leads to that outcome. Feedback loops, delayed outcomes, and other aspects of technical debt fit neatly within the frame of systems theory, which can help us describe the situation in practical terms, anticipate the likelihood of particular outcomes, identify the most important elements of the problem, and find solutions within the messiness of a complex situation.
To reach these solutions, we have to take the following steps:
- Identify some of the most important systemic causes of technical debt.
- Describe some of the most important systemic effects of technical debt.
- Depict how these causes and effects work together within the system of software development and delivery.
- Deciding how detailed the model of the system needs to be, to effectively address the problem.
This list of steps describes the outline of this white paper. We will include a very cursory introduction to systems thinking, which is no substitute for a deeper dive into this subject. While systems thinking is largely unfamiliar to software professionals,[4] the subject is not too arcane, and there are plentiful resources available.[5]
The causes and effects of technical debt
Systems of software innovation create technical debt
There are many roads to technical debt. Some are fairly immediate and direct: for example, a developer may copy and paste code, simply to save time. Others are more distant and indirect: the reasons why the developer needs to save time, such as a company mandate to finish a project by an aggressive deadline, are just as relevant to the creation of technical debt.
Once you take a systems perspective on technical debt, you need to take into account both the direct and indirect causes. The same indirect cause may lead to multiple direct causes. For example, hyper-aggressive deadlines may lead developers to look for a broad range of shortcuts – basically, anything that can save time, even if they know that they’ll pay for their haste later. A different indirect cause may lead to different direct causes. Even under deadline pressure, experienced developers may still occasionally measure technical debt, if for no other reason than to see how heavy a burden they are creating. If the chief source of technical debt is the relative inexperience of the team, then team members might not see the need to measure technical debt at all, or even know how to take that measurement.
Figure 1: Indirect and direct causes of technical debt.
We will use this distinction between direct and indirect forces to organize our list of systemic causes. For each direct cause of technical debt, we will cite many of the indirect causes that lead to it.
An important note: Rather than list every conceivable systemic source of technical debt, we will focus here on some of the most common ones.[6]
As you may have already noticed, the same indirect cause may lead to multiple direct causes. For example, in our list, Management and customers do not see the value in cleaning up technical debt leads to two direct causes, Pressures make shortcuts in coding necessary and Teams don’t commit to cleaning up technical debt. This “over-determination,” in which the same cause has multiple effects, or the same effect has multiple causes, is typical of complex systems like the organizations that develop and deliver software, and is, therefore, one of the chief reasons we have taken this systems perspective on technical debt.
Technical debt hurts systems of software innovation
Now that we have seen some common systemic sources of technical debt, we can examine the effect that technical debt, in turn, has on the system of software innovation within an organization. As we shall see, the costs of technical debt go far beyond the number of extra hours needed to develop new code or the number of hours needed to eliminate technical debt through refactoring.[7]
When discussing the effects of technical debt, people tend to focus on the most immediate and noticeable one: teams become less productive, because every work item or bug fix requires additional work. The effects are actually much greater: not only does technical debt hurt teams in other ways, but it also damages activity at the value stream, and organizational levels. As we will see, the effects can be subtle and insidious, such as the way in which technical debt complicates decisions about the software portfolio. Others are much easier to identify: for example, technical debt can shrink the valuation for a startup software company looking to get acquired.
Important note: Once again, we are not trying to create a comprehensive list, but instead a collection of some of the most common consequences of technical debt.
Putting it all together
For some software professionals, enumerating lists of causes and effects of technical debt is helpful, but not enough to build a strategy for reducing and preventing technical debt. Which of the causes is the most common? Regardless of what happens in other organizations, which cause is the most important for us? How can we paint a picture of how technical debt is hurting us that people throughout the organization can understand?
For these reasons, we will take the next step in treating technical debt as a systemic problem. We will use the concepts and tools developed for understanding the behavior of systems in general to address the conundrums of technical debt. We will show how to use systems theory to depict the system as a whole, including the interactions among the parts of that system that contribute to technical debt, or suffer because of it.
Painting a systemic picture of technical debt
Treating technical debt as a systems problem has its greatest value at three moments:
- The identification of the most significant causes and consequences of technical debt.
- A plausible estimate of how likely different strategies for dealing with technical debt are to succeed.
- A more informed conversation among the people who contribute to, or are affected by, technical debt.
To make these moments a reality, we have to delve a little more deeply into systems theory.
From a perspective to a model
As we have seen in the previous discussion, systems analysis provides a language and a mindset for describing the technical debt problem in meaningful and precise terms. Systems modeling is the chief tool for creating this picture: the model tells us not only what aspects of the system deserve the most attention (for instance, pressures on the team, the average skill of developers, etc., lower productivity, etc.), but also how they interact. These interactions are as important as the system components themselves.
Why should you learn systems thinking, or the special language of systems modeling?[8]
- Dynamic systems are more than a list of causes and effects. A list of causes of technical debt is only useful to a point. The list is like a box that contains pictures of eyes, ears, a mouth, a nose, a hand, and other pieces. Only when you assemble the whole picture can you see the Mona Lisa.
- Systems modeling identifies the critical factors behind problems. A list of causes does not tell you which among them is the most pernicious source of technical debt. A systems diagram, on the other hand, can show you that the same cause contributes to multiple effects, making it an obvious first target for technical debt reduction efforts.
- Systems modeling avoids scapegoating. One of the most important outcomes of systems analysis is that it reduces the amount of unfair finger-pointing. Individuals may want to be “good citizens” about technical debt, but the system in which they work makes it difficult for them to do the right things.
- Systems analysis provides a common, well-established, well-proven language that everyone in the organization can understand. Executives, business users, and other people who inadvertently contribute to technical debt, or feel its effects, have little patience or interest in learning about cyclomatic complexity or naming conventions. They can understand a relatively simple picture that shows how, for example, over-commitment encourages technical debt, which in turn undermines the ability to meet future commitments.
A simple model of an over-committed team
Over-commitment will be the starting point for our brief primer on using systems modeling to understand technical debt. We will start with a very simple system that consists of two “stocks,” a systems modeling term for quantities that we want to track within a system. In this case, the stocks are a backlog (unfinished work) and code in production (finished work). The rate at which work items move from the former into the latter is, of course, the development team’s velocity. If the world were as simple as this diagram, the team would steadily move work items from the backlog into production at a fixed rate (with some natural, minor fluctuations.
Figure 4: A simple model of a software team.
Of course, the world isn’t that simple. We will introduce two variables into this picture:
- Deadline pressure. The team is under pressure to meet some highly aggressive targets. In fact, the team is overcommitted: It cannot meet these targets without cutting corners.
- Technical debt. Cutting corners means accruing technical debt. Once the deadline pressure exceeds the team’s ability to meet its commitments without accruing technical debt, additional pressure produces technical debt at a higher rate.
To illustrate the connection between these two variables, we draw an arrow between them. The plus sign (+) indicates a direct correlation between these variables: as deadline pressure rises, so does technical debt.
Figure 5: The deadline pressure component of our sample model.
Now we will connect technical debt to the work of the team. We draw another arrow between technical debt and the team’s velocity. We use a minus sign (-) to indicate an inverse relationship between these two elements of the system: as technical debt goes up, velocity goes down.
Figure 6: Deadline pressure and technical debt added to the model.
Finally, we will add a third relationship. By drawing a line between velocity and deadline pressure, and marking it with a minus sign (+), we are saying that deadline pressure pushes the team to increase its velocity. However, over time, as technical debt mounts, velocity will actually decrease.
Figure 7: Further additions to the model, showing a positive feedback loop of lowered velocity, higher deadline pressure, and greater technical debt.
Here, we have drawn a simple picture that would make sense to programmer and customer alike. After a certain point, the more pressure the customer puts on the development team, the less productive the team becomes.
This simple model leads to some even more dark conclusions. There is an unspecified limit to the number of corners that the team can cut. Therefore, further pressure is meaningless, once the team is maximizing technical debt. Velocity will continue to drop, however, so the system never reaches a stable equilibrium. There are only two ways to fix the damage that this positive feedback loop creates: (1) removing the pressure to give the team enough time to clean up technical debt to an acceptable level; or (2) pulling the plug on the project. Unfortunately, stakeholders often see both alternatives as unacceptable, which lets this system continue to spin out of control. Eventually, circumstances will force one of these decisions on those stakeholders, at a much higher price tag than if they had intervened earlier.
Identifying destructive relationships
Systems modeling helps to identify risks, often seen in destructive patterns of behavior that get triggered when the system enters a particular state. In the diagram below, team discipline about good coding practices is the bulwark against technical debt. Once outside pressures damage that discipline, a nasty positive feedback loop starts. Technical debt mounts, making it harder to work in the code. Team morale drops, making people more cynical about their work, further eroding team discipline.
Figure 8: A different model, incorporating team morale and discipline into the picture.
Turning a model into a simulation
We could take an additional step in modeling a system like our “iron triangle” of deadline pressure, technical debt, and team velocity. Using a tool to simulate how the system works, we can see what outcomes result from our decisions, such as taking a hard line against technical debt. We would need to add some specifics, such as the team’s velocity, the impact of technical debt on that velocity, the size of the backlog, and so on. Once we do, we can use an open source or commercial systems modeling tool to watch the system in operation. If the model is simple enough, we can also create a game, based on the model, which we can use as a test bed for different strategies for dealing with technical debt.
Simulation can lead to further conclusions that have important practical consequences. For example, you might discover that there is a take-off point for an uncontrolled spiral into massive technical debt. Once deadline pressure rises to a predictable level, the team will be cutting corners at an increasingly frantic rate. Learning how to avoid this death spiral would be one of the key lessons from just such a simulation.
Simulation also creates a more realistic sense of the causes and consequences of technical debt. For example, a simulation may show you that some countermeasures, such as code review, may have more or less impact than expected. Running the simulation several times also reduces the chance that one particular outcome, either in the simulation or in real life, represents an unlikely outlier event, not the most likely outcome that you might expect to get from, for example, budgeting time in each sprint for technical debt reduction. Simulations also show the delayed effects, such as when the time budgeted for technical debt reduction needs several sprints to start really paying off.
Deciding on the essentials
Systems analysis can generate complex models easily. Fortunately, you don’t need that complexity to make good use of systems thinking. Every model tells a story, and that story won’t be the same everywhere. In one organization, the root cause of technical debt might be deadline pressure; in another organization, it might be poor engineering practices. In one organization, the chief cost of technical debt might be reduced velocity; in another, it might be unreliable plans.
Systems analysis provides a canvas and a set of implements for making a precise description of how technical debt hurts your software innovation efforts. While it can be valuable to see, in operation, all the causes and effects that may be important, an all-encompassing model does not tell you which causes and effects have the greatest relevance in your organization. Therefore, you have to use systems analysis to tell your own story.
That being said, a complex model that describes the common relationships in a software innovation system struggling with technical debt may suggest elements of your story that merit greater emphasis. For example, you might not consider team morale to be a critical factor, until you see a model in which team morale matters enormously for both preventing and reducing technical debt.
For that reason, we have provided a more complex model for your edification. Since it has many moving parts, we have written a companion document to explain the components and relationships that comprise the model. For more information, see Thierry’s model. [9]
Patterns, principles, and archetypes
The more complex model has an additional benefit: it contains patterns that are common in organizations struggling with technical debt. The two types of patterns that emerge from this type of analysis, systems principles and systems archetypes, will both help to master technical debt.
Systems principles are general guidelines for understanding how systems work. By extension, they are ways of developing realistic expectations about dealing with systemic problems like technical debt. Here are a few examples:
- Systems behave in unexpected ways. To return to an earlier example, the usual way to measure the cost of technical debt is lower productivity. There may be other consequences that are more important, such as the damage that technical debt can inflict on sprint and release planning, portfolio management, and flow. Unless you looked at technical debt in light of the larger system of software innovation, you might miss these bigger costs.
- Causes do not have linear effects. For example, people facing severe technical debt burdens sometimes lament, “If only we had better developers.” Hiring top-rank developers, however, might not bring the relief expected. More talented developers might help avoid technical debt, but not help dispel the existing debt. Cleaning up other people’s debt is a thankless chore that some top-notch programmers might prefer to avoid, reducing the value of investing in them.
- Causes do not have immediate effects. The delay between cause an effect can lead to all kinds of false conclusions. For example, instituting code review might not have a significant impact on technical debt for several sprints, until new disciplines take hold. Delayed effects often lead people to over-correct when they appear. For instance, a few sprints after implementing an open source module, the team might experience a reduction in velocity. The problem might not be technical debt in that module, but just the team’s inexperience working with that unfamiliar code more frequently.
Systems archetypes are the product of past experience analyzing other systems, then anticipating certain behaviors of the current one under the microscope. In many cases, systems analysis can identify these patterns across systems that superficially appear to be unlike, but share some important commonalities. Here are a few examples, again showing their relevance to technical debt:[10]
- The tragedy of the commons. When people who use or contribute to a resource do not share responsibility for its maintenance, the system suffers from neglect. You often see this problem in development teams off-load maintenance work to other teams, in which case, the perceived responsibility for preventing technical debt is much lower.
- Limits to success. In this pattern, success leads to a limiting condition that blocks further progress. For example, a team might negotiate with its management permission to devote some time to technical debt reduction. However, after reducing technical debt measures by a noticeable degree, management pulls the plug on further efforts. Unfortunately, both management and the team were looking at the wrong measure. Instead reducing the number of copied code blocks and overly complex classes, the real measure of success was increased team productivity.
- Drifting goals. Instead of trying new approaches to addressing a problem like technical debt, people may lower the bar for success. This pattern is especially common in dealing with technical debt, which requires some experimentation to discover the best collection of techniques to both prevent and reduce technical debt for a particular code base, team, and organization. After trying a single approach and not getting the desired results, teams may retreat from their technical debt containment goals.
Conclusion
When faced with a problem like technical debt, the temptation is to take linear, direct, and narrowly focused approaches. If only we weren’t so sloppy in writing code…If only we spent more time mentoring our junior developers…If only we had less pressure on us… By treating technical debt as a systemic problem, we are more likely to understand its causes, and deal with its effects.
- The major causes and effects are identifiable. Most of the causes of technical debt are not as immediate as the callous disregard that a careless developer takes when writing code. The real causes lie elsewhere, in the team, the value stream, and the larger organization. The real consequences lie in these other planes, too: while we might sympathize with the plight of an unhappy programmer trying to work with technical debt-ridden code, we suffer substantial financial consequences of technical debt-ridden parts of our software portfolio.
- By plugging these causes and effects into a system model, you can better assess the problem and plan next steps. Until you create a picture of the system of software innovation, it is difficult to tell how effective corrective measures will be. While some degree of code hygiene is always a good thing, taking a particular step, such as running regular static analysis tests, or implementing code review, may not be as effective as other measures.
- Systems thinking naturally involves people outside the team in addressing technical debt. “We need time to clean up some problems with the code, so you’ll just have to trust us,” is a far less effective message than, “Here is the picture of how you unwittingly contribute to technical debt, and the hidden costs of living with it.” The picture of the whole system, not just the report from a tool like Sonar, is the real starting point for discussion of how to deal with technical debt.
Appendix
Tables and figures
The following tables and diagrams appear in this document:
Figure 1: Indirect and direct causes of technical debt. 3
Figure 2: Common causes of technical debt. 4
Figure 3: Common costs or consequences of technical debt. 5
Figure 4: A simple model of a software team. 7
Figure 5: The deadline pressure component of our sample model. 7
Figure 6: Deadline pressure and technical debt added to the model. 7
Figure 8: A different model, incorporating team morale and discipline into the picture. 9
[1] For a full explanation of technical debt, see “Introduction To The Technical Debt Concept” at https://www.agilealliance.org/introduction-to-the-technical-debt-concept/.
[2] Therefore, in this paper, we will focus on the sources of technical debt that result from a lack of disciplined coding, recognizing that some marginal technical debt creation happens in any case.
[3] Dice Of Debt is available at https://www.agilealliance.org/dice-of-debt-game/.
[4] Two of the most notable applications of systems thinking in software development include Gerald Weinberg, An Introduction To General Systems Thinking, and Grady Booch, Object-Oriented Analysis And Design With Applications.
[5] Some good introductions to systems analysis include Donella Meadows, Thinking In Systems: A Primer; George Klir, Facets of Systems Science; and the Systems Thinking Wiki at http://systemswiki.org/index.php?title=Main_Page.
[6] For the most immediate measures of technical debt – how each type of code quality problem contributes measurably to a loss in productivity – see the Agile Alliance Debt Analysis Model (A2DAM) at https://www.agilealliance.org/the-agile-alliance-debt-analysis-model/.
[7] For a good discussion of how to budget time for refactoring, see Greger Wikstrand’s blog post, “Budget For Technical Debt Reduction,” at http://www.gregerwikstrand.com/technical-debt-reduction/.
[8] For an overview of systems modeling, see Hiroki Sayama, Introduction To The Modeling And Analysis Of Complex Systems.
[9] The more complex model includes elements that do not appear in the simple model we developed here in this document. Therefore, expect to see a different set of assumptions and relationships when reviewing that model.
[10] One of the most familiar applications of systems archetypes to business challenges can be found in Peter Senge’s The Fifth Discipline: The Art And Practice Of The Learning Organization.
