(See the original essay on Scrumban.)
The Meaning of Scrumban
If I could boil the essential idea of Lean production down to a single sentence, it would be this:
Continuous improvement and continuous delivery are complementary practices.
Continuous delivery creates the necessary operational conditions to sustain continuous improvement. But continuous delivery itself can only be achieved by an iterative process of improvement.
If you believe that continuous improvement is important to your business goals, then your first concrete goal should be the establishment of a continuous delivery system. Continuous delivery is not something you can buy off a shelf. No consultant can provide it with a magic incantation. Achievement of continuous delivery must be a process that is specific to the product and the resources available. It must be achieved in stages, and this process is, itself, the beginning of continuous improvement. Only the goal of continuous delivery makes this iterative process possible, and only once it is achieved can continuous improvement be sustained.
Scrumban was intended to be a practical demonstration of achieving a continuous delivery process by means of an iterative improvement process. Since the arrival of Lean production to economies outside of Japan, there have been many such demonstrations across many industries. Scrumban demonstrated such a process within the field of software development.
An iterative process improvement project must have a starting point, and for Scrumban, that starting point is Scrum. Scrum is not a good process, but it has two qualities that are useful for our purposes: it is simple, and it is popular. And when it is done as well as it can be, it features one critical ingredient that we can build a Lean production process around: a multi-disciplinary team. So, despite not being a good process, Scrum is still a good starting point for continuous delivery and Lean production.
Scrumban is not a set of practices. Scrumban is not a prescribed process. There is no canonical reference implementation of Scrumban. Scrumban is not an approach to “doing Kanban.” Scrumban uses kanban in pursuit of continuous delivery and continuous improvement. Scrumban draws from the body of practice of Lean production and related methods like Theory of Constraints in pursuit of continuous delivery and continuous improvement.
Scrumban is a deployment strategy for continuous delivery. There is no single concrete process, other than Scrum, but there is a regular meta-process: Scrumban revolves around the standard Plan-Do-Study-Act loop of continuous improvement. Scrumban is PDSA with a starting point of Scrum and a goal of continuous delivery:
- Implement a standard Scrum team with the usual practices. It is essential that this is a multi-disciplinary team, capable of delivering complete, deployable builds. Multi-disciplinary does not mean 5 different kinds of developers. There is some implicit division of labor where designs are verified by someone who is not the designer.
- Change one practice in the direction of continuous delivery. This change may increase iteration frequency. It may reduce cycle time or variation in cycle time. Improvements to quality reduce variation of cycle time. Divisions of labor improve cycle times. Reductions of work-in-process improve cycle times. At some point, there will be tension between cycle time and release frequency. You will eventually abandon one or more Scrum practices to resolve this tension.
- Stabilize the current process under the criteria of statistical process control.
- If you have achieved a reliable state of continuous delivery, then congratulations, you have escaped from Scrumban and are now practicing Lean production.
- Repeat step 2, indefinitely.
The immediate object of Scrumban is the Lean production of software. The ultimate object of Scrumban is continuous improvement. If continuous delivery is a necessary condition to achieve continuous improvement, then Scrumban is a strategy to achieve continuous delivery. The tools that Scrumban applies to this effect draw from Lean production. Lean production is a fruitful approach to achieve continuous delivery. Achievement of the Lean production of software makes the realization of continuous improvement possible. Any viable path to continuous improvement is a worthwhile path. Scrumban is known to be a viable path. Scrumban can be realized by starting with Scrum.
Scrum is a natural starting point for Lean software development because breaking functional departments up into small multi-functional units is the most conventional and time-honored approach to implementing Lean production. Advice to “start with where you are” and preserve functional departments is entirely contrary to the long tradition of Lean production. Introducing kanban is traditionally regarded as a later step in implementing continuous delivery, and premature introduction of kanban was considered to be a novice mistake by the architects of Lean production. Beginning with Scrum installs essential foundations before adopting more advanced practices like kanban. Scrum is not the only way to achieve this, but it is an easy and convenient way to achieve this.
The Origin of Scrumban
The Scrumban method did not appear like a bolt from the sky. It was one evolution of established lines of thinking extending back decades. Above all, Scrumban is a descendant of the Microsoft Feature Crew methodology circa 2000.
By the early 2000s, it was already becoming apparent that the center of gravity of the world of software development would shift away from packaged software and toward Internet-based services. Continuous delivery was the inevitable endpoint for such service-oriented development. Not everybody at Microsoft had received the memo yet, but some of us could see where things were going.
From where I was sitting, the emerging Agile methods of the day felt like small-scale solutions to yesterday’s problems. The ostensibly novel practices of Extreme Programming were years behind what developers at Microsoft were doing every day. The most innovative part of early Agile seemed to be the intensity of the propaganda. It was clear to me by 2001 that there would have to be additional progress to deal with the emerging problems of continuous delivery on the Internet. It was also clear that not all of this progress would come from within the world of computer science or software engineering. Harlan Mills and Tom Gilb and E.W. Dijkstra had pieces of the solution, but none of those things were organizationally minded at the speed and scale required. Some parts of the solution would have to come from outside or be invented anew.
Too much of the software engineering literature of the time was focused on the work of individuals or small teams. But there was a parallel robust literature in product development and systems engineering that dealt more thoroughly with issues of scale. Quality Function Deployment evolved into Design for Six Sigma. This stuff was about building jet fighters, power plants, and medical instruments. Serious business at serious scale. The software business circa 2000 suffered from horrific systemic quality problems. As much as any organizational immaturity, poor quality was holding the industry back. These heavy systems engineering theories placed quality front and center, within a mature statistical framework of understanding. Not the bug-count and beta testing mentality of software development.
The downside to so much of this product development literature is that it relied upon archaic plan-driven phase-gate project management theories. Here was one area, at least, where the software people seemed to have gotten ahead.
In the summer of 2003, I tried my hand at reconciliation and proposed a model for Agile Design for Six Sigma. I argued there was nothing intrinsic about large batches of work or documentation to any of the practices of DFSS. I argued that a pipeline of work could be any arbitrary length and still delivered in small increments and that all phases of design could be conducted simultaneously. And above all, that the low-coupling requirements of Axiomatic Design that were essential to product robustness were also the enabling condition for evolutionary delivery.
I chose Scrum as the project management framework for this process, partly for the simplicity of explanation and partly for the shock value. Scrum was not an ideal fit for the requirements. Fitting a ten-week pipeline into a four-week iteration was a difficult circle to square. But this very problem set up an essential tension that required a solution.
The key to resolving this conflict came in the form of Donald Reinertsen’s theory of Lean product development. I approached these ideas from a long background in the world of manufacturing. His arguments made intuitive sense to me, and it led me to return to the work of Goldratt and the literature of Lean production in a different light.
Then I realized that a solution was right in front of me. A fully realized large-scale Lean software production system was already in operation at Microsoft. Office and Visual Studio were shipping products using Feature Crews. This was not an obscure research team out in the wilderness. This was hundreds of people, delivering the lifeblood of the company, right then and there. With Feature Crews, you could have a ten-week pipeline and still deliver a new feature every week. Feature Crews satisfied the needs of Agile DFSS even if they didn’t know it and weren’t attempting it.
When my own team ramped up for a new project in Spring 2004, I balked at using Scrum for the opposite of the usual reason: I felt that Scrum was already obsolete, a warmed-over skunk-works philosophy, a relic of the 1980s. I could not tolerate another round of sprint planning, and I was too well-versed in software engineering not to know that the Scrum approach to estimation was bad practice. (Good estimates are based on data, not feelings.) I proposed a modernization using Scrum roles but a scaled-down Feature Crew scheduling model: a compact generalist team with a defined workflow, limited work-in-process, continuous delivery, and event-driven planning. Before Personal Kanban, there was the Personal Feature Crew, and this was the original Scrumban.
In 2005, I worked out a generalized framework to transform any development project into a continuous delivery model. Late in 2006, my friend David invited me to help him do just that. I was able to join him early in 2007, and in the Spring of 2007, we rolled out the second Scrumban project. This was the project that most closely resembles the process described in the 2009 Scrumban book which became the popular understanding of the concept.
Scrumban was never intended to be a defined, prescriptive process. It is a demonstration of a way of thinking about cooperative knowledge work. In 2007, it was unusual to speak about software development using terminology like throughput or inventory. Today, it is common to hear this language, and normalizing it was the primary goal of Scrumban and the early kanban methods. Scrumban was also not meant to be the final word in organizing for Lean software development. Scrumban demonstrates the relationship between continuous delivery and continuous improvement. Continuous delivery and continuous improvement remain the eternal goal.
Scrumban was proposed under a moral foundation of open exploration, without gatekeeping. Scrumban is not proprietary. There is no official certification for Scrumban that carries my endorsement. Despite its history, Lean production is not the property of Toyota. Lean software development belongs to its practitioners, and so does Scrumban.
Kanban-based methods like Scrumban succeeded because they solved a real problem, the problem of continuous delivery, a problem that competing methods were poorly equipped to address. But continuous delivery has deeper importance than its immediate practical value. Continuous delivery is the key to unlocking continuous improvement. This is the lesson of the Toyota Production System.
Lean, Deming, and the Toyota Production System should matter to all of us because they are built upon a stakeholder theory of production, where people are treated as ends in themselves, and purpose is a motivation on a footing with profit. Lean software production shows that this way of thinking belongs in our present as well as the future.
Note: This is an excerpt from the forthcoming book “Scrumban Zero” by Corey Ladas, reproduced here with kind permission from the author.
