As we wrote a book on succeeding with distributed agile teams, we discovered that many of the same agile principles and practices could apply to our writing. We further learned that the exercise of pair-writing could help teach some of these same concepts to distributed teams who may struggle with applying agile principles.
1. INTRODUCTION
For years, we’ve heard that only collocated teams can use agile approaches. This disregards the roughly 50% of agile teams who already are distributed or dispersed [1]. Some of these teams have been successful. Instead of assuming agile teams can only be collocated, what if we wrote a book to help distributed and dispersed teams see options for their agile success?
We found that many agile principles and practices enhanced our teamwork and created an outstanding project environment despite the fact that we were not collocated. Some of the practices were: pairing, continuous integration, write instead of discuss, sustainable pace, feedback about the work and how we worked together, and more.
If your distributed agile team has trouble progressing, consider asking people to write in pairs. The writing might break the logjam for the team’s product development. It may also inspire your fellow developers, testers, product owners and other team members to apply agile principles and practices in a distributed environment.
Some people say they “hate” writing. In our experience, it’s not the writing people object to; it’s the potential audience response. Instead of framing writing as part of the project work, the writing is a vehicle for exploring and learning collaboration skills, pairing and sharing domain knowledge. Yet, it is likely still valuable to share the output of the exercise with the team to see if you are improving.
If you are part of a distributed team, as a team member, coach, or manager, consider the principles we used, and see if they will help your team.
2. Background
At the Agile 2017 conference, Johanna and Mark walked together to the last morning’s sessions. Johanna was thinking of a geographically distributed agile teams book. She knew Mark had extensive experience with distributed agile teams.
We compared notes. We noticed very few sessions addressed the problems on distributed agile teams and the information we saw was outdated. We decided to try to write the book together. We agreed we would pair-write (as opposed to ping-pong) and that we would experience many of the same challenges and benefits as distributed agile teams.
Too often, we hear that distributed and dispersed agile teams require each person work according to his or her functional role. We soon learned that because we paired, we had much more flexibility in our roles on this project. Together, we focused on our collaborative work. Separately, we contributed as we could to move the project forward.
As we co-wrote the book, we realized we were living the same principles as successful distributed and dispersed agile teams do. We worked in short timeboxes and reflected often. We chose to write together, rather than separately. We did choose to create images separately so we could then discuss them together.
3. Our Story
We started working on the book August 30, 2017.
We started to form and storm as a pair during September. However, Johanna had international travel; Mark lived through a hurricane. Those events caused us to delay some of our work.
We managed to overcome those delays and continue exploratory work as a pair up until October 19, 2017. This work was similar to “spikes” that a software team might pursue to explore technologies and early designs in a software project.
During that time, we developed what we called a “book frame” which was a story map for the book. We knew who we were writing for (e.g. personas).
In late October, we started the first chapter. We discussed and wrote many of the principles and created the images. That work generated placeholders for the second and third chapters. We wrote where we had the most energy around the particular topic, moving between chapters. (Writers don’t necessarily work in a linear fashion. We did not.) That third chapter, “Communicate to Collaborate” generated a ton of ideas which suggested new chapters.
As we reflected on this part of the writing experience, it was not unlike a software team building the skeleton of a new application. They may develop parts of the system and mock others to establish basic functionality and concepts.
Johanna then had three weeks of travel (late October to early November), which meant we encountered the same time zone and distance problems as every other distributed team does. The first ten days, Johanna was unavailable to write. The last week, Johanna was in Israel, a six-hour time zone difference. We only had one hour a day to write. And, even though Johanna had (supposedly fast) wifi at her hotel, we could not use both the audio and video of Zoom.
Again, this experience was similar to team members who might relocate or even be asked to multi-task across multiple projects. In our case, we checked-in with each other each day to adjust how we worked based on the time frame and conditions we had for the day. The goal was always to move the work forward.
Once we were back in our home offices, we worked on the first three chapters “simultaneously” throughout November and December of 2017. We spun off ideas into other chapters. We reordered chapters and continued. We wrote the next two chapters relatively quickly in January and February of 2018, bring our “finished” work to five chapters.
We have seen a similar pattern in the distributed (and collocated) agile teams we have coached. As team members learn about each other’s strengths, their availability to collaborate and can establish a rhythm, they can start to make tremendous progress on core functionality.
We finally felt as if we’d finished the first five chapters in early March 2018. We decided (Johanna pushed, Mark agreed) to create an early version that we could release and get some feedback on. We spent March “finishing” the chapters so we could move to Markdown and use Leanpub to release an early version.
We’d used continuous integration in Google docs, but because we needed to move to Markdown, we had a brief staging period. Johanna tried several different export tools and decided on one. She then spent about an hour per file creating Markdown that would compile into a book. In this case, our “final” integration took less time than software teams, but it still slowed us down. Once we moved a file to Markdown, we could still collaborate in Google docs, but we would not re-export. We highlighted changes and one of us moved the new text to the Markdown file.
This slowdown and this shift in emphasis to continuous deployment allowed us to think about how we could begin getting feedback on our reviewable product, even if it was not final. We wanted to invest the time to develop our deployment pipeline so that we could improve it as we complete the book. As of this writing, we still have some manual steps, but we are far closer to push-button deployment to production in Leanpub. This will allow us to incorporate feedback from our “users” (early reviewers) as quickly as we wish.
4. Problems and Discoveries
Every project encounters problems and ours was no exception. We discovered alternatives that worked for us.
4.1 Tooling Challenges
Our first problem was the writing tools. Like most new software teams, we needed to establish the tools that would work well for both of us (as a team of two).
We decided to self-publish this book for a variety of reasons. We chose leanpub.com as our publishing and writing platform, which means we needed to write in Markdown.
Johanna has her tools for Markdown writing (TextMate with specific plugins). However, we could not literally pair-write in TextMate, using Dropbox as our storage. We experimented in these ways:
- Writing in TextMate and sharing the screen. The other person felt disconnected.
- Writing in TextMate, using a timer, and changing the writer. On reflection, Mark does not remember the timer and Johanna probably ignored it.
- We tried several other Markdown tools, but they didn’t provide the real-time pairing that we wanted.
We had each worked in Google docs before as a pair, so we tried that. The writing worked. Johanna found a Chrome add-on that is supposed to work to convert docs to Markdown. We decided to take the chance and use Google docs.
The benefit of working in Google docs, beyond the real-time pairing, is that we were not dependent on a screen sharing tool. This became important later in some more challenging pairing situations.
As we wrote, we discovered humorous quirks with Google docs. We termed those quirks, “Google knows best.” For example, sometimes Google would want a hyphen where we did not feel one was needed. Sometimes we bent to Google’s will and other times we felt more strongly about the suggestion and ignored it. Those experiences reinforced our rule, “Don’t let the tools drive your writing.”
4.2 Finding Hours of Overlap for Collaboration
We were able to quickly work out our hours of overlap when we could pair together. Fortunately, both of us work on the US east coast. So our time zones aligned. However, with Johanna’s consulting and Mark’s work as a full time internal coach with a rapidly growing distributed organization, finding openings in our calendar was challenging.
Distributed agile teams, especially when they first start to work together, may have this problem, too. If the team members are supposed to work on multiple projects at the same time, they may have trouble finding overlap time at all. Hours of overlap, in our experience, is key to the success of distributed agile teams. The more hours of overlap a team has, the easier it is to collaborate on the work.
Because we are in the same time zone by default, we should have had many hours of overlap. However, we each had several other projects underway. With our other work, we could only find small chunks of time to work together and pair-write. We are NOT recommending a team work on multiple projects at one time, but we realize this happens. A team with multiple projects should shift or release some of their workload to other teams so their available hours of overlap increase and allow for more collaboration.
What we found was the magic of 30 minutes. We decided that if we could only meet 30 minutes a day, that it would be sufficient to make progress on the book. This became one of our biggest assets in quickly assembling the story map and writing each chapter. When you only have 30 minutes to focus, you can be very intentional about what you want to accomplish with your co-author. There is no time for waffling on what to write.
Initially, we found our 30-minute time slots early in the morning, but this needed to change from time to time. We both work from home and juggle multiple work projects and family needs. Some weeks Johanna had consulting or classes. Other times, Mark had meetings with various teams within Sonatype or had to take his kids to and from school and events.
There were other times where we needed to make larger changes in our calendar. If one of us were at a conference, we both realized a writing session wasn’t feasible for one or more days. That would be a rare case where we might work on something solo such as images or rewriting a section of a chapter that we had discussed. However, we always wanted to re-establish our rhythm of 30-minute working sessions as quickly as we could.
When Johanna traveled out of our time zone, we tried to work together. We had scheduled time when she was in Israel to write. However, the wifi was not sufficient, even just for an audio connection. In this case, we were able to use the text chat within Zoom or Google docs and pair-write in Google docs, but it did not have the same immediacy in terms of communication richness or naturalness we were accustomed to.
Other times, when the network bandwidth was sufficient, all we needed to do was adjust when we convened our 30-minute work session.
4.3 Discovering our Always Available Backchannel
We had communication tool problems, also. We used Zoom for audio and video. We had times when our cameras didn’t work, and our headsets didn’t work. Zoom worked—we didn’t. In those cases, we were able to fall back on texting each other’s cell phone. We used text as a backchannel.
We’ve both tried to collaborate at distance for many years. We had plenty of tool problems, that prevented easy collaboration then. The advent of Google docs and Zoom created an environment for us that made our collaboration easy, as opposed to difficult. The benefit of these technologies is that they supported collaboration without getting in our way. The tools faded into the background, leaving us to focus on our work product together.
In our book, we define the concept of a backchannel as a default channel to get a message to your partner or team about when and how you are available to collaborate. The backchannel is an “always accessible”, low bandwidth, text-based communication channel. Typically, a backchannel helps the distributed team provide context and coordination when other channels are not available. (This channel is different from people being always available for video, or other kind of interruption-based communication.)
We used SMS text as our primary backchannel. It also became useful when we lost our primary communication channel of audio and video and needed to coordinate on how to “reconnect” during a pair writing session.
If one form of the backchannel didn’t work, we would quickly switch to the other. We usually resolved any coordination issues in 10-15 minutes. If problems happened during one of our pair-writing sessions, we would resolve it in less than 5 minutes using our backchannels.
Distributed teams need backchannels, too. We deliberately chose an always-text tool separate from our meeting tools, so we could discover and recover from meeting problems. Distributed agile teams need the same kind of a backchannel so they, too, can recover from communication snafus fast.
4.4 Learning Each Other’s Cues
Every team needs to learn how to work together. We were no exception. Once we got past the meeting times and tools, we needed to learn how to write with each other.
We quickly learned how to signal each other with challenges we experienced. Here are some examples:
- “You have to say more about that” – this was typically a signal from Mark to Johanna that what she just wrote may not have been clear to her co-author or other readers.
- “When I’m not so tired, I’m pretty human” – We both suffered from insufficient sleep (or illness) at various times. Johanna, when she’s tired, tends to be less than compassionate and empathetic. At one point, Mark looked at what she wrote and looked at her. That’s when she said the human phrase. After we both laughed, we could rewrite the less-than-human approach to something more reasonable.
- “Mind if I tweak that?” – we are both experienced writers. And, Johanna is allergic to passive voice and adverbs. We both used this phrase to “unpack” each other’s writing.
- “We need to unpack that.” – Sometimes Johanna would write something brilliant that only she could understand. Mark realized it would have potential, but we probably needed to unpack the concept from one complex sentence to anywhere from one to four paragraphs. We did a lot of unpacking.
- “I’m okay with that” – Often, as we pair wrote, one of us would add or slightly tweak a sentence as the other person was completing the rest of a paragraph. When we made changes solo, we highlighted the change and checked in at the next pairing session with the other person to make sure “they were okay with that”.
- “I found where you are (when jumping around in a Google doc)” and “Click where you are” – When we first started to use Google docs, we did not realize we could click on the other person’s icon and jump to where they were in the file. We scanned the document to see the other person’s cursor when they clicked or highlighted. Once we learned to click on the other person’s icon, this became much easier and we let the person know “I clicked to where you are”.
- Comments, highlights and XX marks the spot for more work – Even on our “best” writing days, we realized we needed references, or a cross-reference in the book. Sometimes, we needed to research and add in more detail later. We used XX as a placeholder so we could write as a pair and then assign one or both of us specific non-pairing todos so we could finish the day’s writing.
- “The writer/typist/AV engineer/secretary didn’t show up today” – Sometimes, we both have trouble with spelling. Sometimes, we had trouble with our headsets or cameras. Sometimes, we were surprised by what our calendars had as available time. We used the phrase, “The writer didn’t show up today,” or whatever role we wanted to refer to. This was our indication to our partner that we might need some help in that area.
5. Our Working Agreements
We needed some working agreements, just as any other agile team does. Here are ours to help encourage a continuous flow of writing:
- Default to writing. It’s very easy for pair-writers to spend their entire timebox talking. However, we wanted to spend our timebox writing. Our primary agreement was to start writing. This is similar to a team discussing the design or architecture. If we try something, we can decide to continue or to postpone or to revamp.
- Don’t write over the other person. If one is writing, one should be reading or researching relevant references. If we have additions to each other’s words (or want to fix spelling), we wait until the other has moved a line or two down, so we don’t literally write on top of the other.
- If you’re typing, don’t go backwards. If you’re the one writing down the words, continue moving forward. The other person can fix spelling after you’re a line or so down.
- Don’t try to write and talk at the same time. Johanna appears to use different parts of her brain to write and to speak. She is not able to write and talk at the same time. And, when she writes, the words arrive differently than when she speaks. Johanna has a strong rule of “Just write.” We used that rule when each of us wanted to tell a story. Sometimes, we wrote the story. Sometimes, the other person typed what the other person said as dictation. (“You tell the story and let me write what I hear”)
- Change driver and navigator often (about every 5-10 minutes) unless the writer didn’t show up that day for one of us.
- We decided on the work of the day based on our availability and the open work. Cadence vs timebox. (We maintained our cadence of writing regardless of the timebox duration that day… despite holidays, illness and hurricanes.)
- If we wrote asynchronously, we highlighted the changes or additional text. We also established the agreement of reviewing these changes at the beginning of the very next pair-writing session. We also used comments to clarify our position for our future discussion or if we decided to re-read a chapter for discussion the next day.
6. Our Integration and Release Challenges
As we completed several chapters, we decided we needed to be able to produce something for feedback and practicing the act of releasing the “product”. Leanpub allows us to create a book as we write. Since we used Google docs instead of markdown, we needed to convert to markdown and then create the book in progress. We decided we needed to experiment with how to push to production.
Johanna experimented with several docs-to-markdown add-ons. She realized we would have to “fix” the output: add the images, organize the book creation file (the build), and verify that what we had in docs and markdown matched. As we write this report, we are still learning and experimenting.
The title of the book required more experimentation. First, we brainstormed a list of titles that reflected the promise of the book and how the promise would be fulfilled. Then we used some online tools to analyze the attractiveness of the title to readers. We narrowed it down to a few combinations and wanted to use the wisdom of our own crowds. We used a survey to help us choose a title for the book, so we could select a reasonable title.
We also anticipate the normal challenges of creating a print version of the book. The layout differs for print and ebooks. Johanna has a layout person she has used in the past and anticipates she will use for this book.
7. Results and Learning
As we pair-wrote, we learned lessons applicable to distributed and dispersed agile teams.
7.1 Check-ins
At the time we met every session, we had brief check-ins. Sometimes, the check-in was about the solo work we’d done before. More often, it was a personal check-in, about our health (we wrote during cold and flu season), weather challenges, our families, even our work and things we discovered at conferences. We used the check-in time (as little as 30 seconds, as long as a few minutes) to synchronize on our context.
We used our synchronization as a calibration tool. We needed to learn enough about each other’s context to do our best work each day. We paid attention to: the work we do together, each other’s work, our families, and our personal lives. Sometimes we were both “ready to work”. Sometimes it was one person writing more than the other. But it allowed us to find our passion for this project and temporarily put all of our other concerns on hold. We could take our time to focus on our timebox of work for the day.
7.2 The Power of a Streak
We learned the power of a streak, to continue writing every day (or as close to it as we could) to finish paragraphs, sections, and chapters. We learned how to use the tools and when to evolve the tools we used. We looked like an agile team, with continuous integration, delivery, reflection and refinement.
As we first adopted our tools and pair-writing process, we reflected and refined daily (during and after a writing session). Once we grew comfortable with our tools and our process, we reflected and refined every few days and never went more than a week.
7.3 Declaring Done
Knowing what done means for a book might be different than done for software projects. We both know people who have been writing a book for several years. (Johanna knows someone who’s been writing a book for more than 20 years.) That book will never be done because as things change in the industry, the writer adds to the book.
As we wrote, we sliced and diced the book frame and the chapter story maps, to evolve the book as we wrote. At the same time, we kept our audience in mind. We knew what we wanted to help that audience learn.
We have different levels of done. We used short timeboxes, anywhere from 15-45 minutes to write. We were done with each session when we finished the timebox. We created story maps for each chapter, which helps us know when we were done with a chapter. We know when we will be done with the book when we have explained the specific learnings (in our book frame) to our audience.
We also wanted to be sympathetic to the needs of our audience. As we discussed the personas that might read the book, we decided that most readers would be busy. They would be either busy developers, testers, managers or executives. With that in mind, another criteria for “done” would be short chapters that we decided to try to limit each chapter to 5000 words. That word count helped us focus on a particular theme per chapter and help focus the learning.
We realized the need for new chapters in two ways: the first was when we wrote our story maps. If we had too many ideas for a chapter, we realized we had a new theme which required its own chapter. The second was as we wrote the book. We have a chapter in draft specifically for managers and their beliefs about distributed agile teams. We took the time to write a series of myths and why those beliefs are myths. At the time of this report, we are not sure precisely where that chapter fits in the book. We gave ourselves the freedom to divert from the book frame and create new chapters when we felt it would serve the reader.
Both of us know how to write “clean”—fixing typos and eliminating passive voice. And, we had the benefit of pair-writing to clean our words and intent as we proceeded. We checked each other as we wrote, to make sure we said what we meant. Sometimes, we stopped to have a short discussion, where we often realized we needed to “unpack” the wording. More often, we clarified the words. We found the same benefit of pairing as technical team members do: constant review and verification that kept us on the right track.
We will need final copy-editing on the book—every writer needs that. However, we are unlikely to need other editing.
7.4 How We Used Cadences, Timeboxes, and Continuous Flow
Many teams start their agile approach with timeboxes of one or two weeks. We did not. We used a timebox for daily writing of 15-45 minutes. We used a cadence of reflection and replanning, and it was not a regular cadence. (See [3] for more discussion of cadence vs. timeboxes.)
At the start of our writing, we reflected often—almost every time we met to plan or write. Once we became familiar with each other’s preferences (and quirks!), we did not need to reflect as often. Sometimes our check-ins were partly reflections.
After we completed the first couple of chapters, we learned how we would move some of the writing around. We planned each chapter as we needed to. Some chapters we planned once and then wrote. Some chapters we planned, wrote a bit, and replanned.
As we wrote, we sometimes discovered we needed some research, which we could assign to each other. We did not research much in our writing time. You might consider our research as spikes, where we investigated an idea to bring to the other for our next writing session. Because we had firmer plans for each chapter, we could let the writing help us discover what we needed in this chapter, and possibly other parts of the book. This is similar to a team that discovers opportunities to refactor their code and does so efficiently by refactoring where they are working.
As we moved into a continuous flow of writing, our goal was to finish a section or two every day. This meant that we did not necessarily plan at the section level every day, but at least each one to two days. Our story map was our reference for what we planned for the chapter and helped us determine our next planning windows. As we finished a chapter, our book frame guided us on when to generate the next story map. While we did not plan on a regular cadence, we used a quick feedback loop of: plan what we needed for a chapter, write that, review and replan (in this chapter and for other chapters), just as agile projects do. We used a double-loop learning approach, where we checked our assumptions, also just as any agile project does.
Our writing is an example of a flow-based project, which negated a need for a strict planning cadence.
We’ve used several words here: a cadence is a periodic event. A timebox assume a regular, strict cadence that defines the start and the end of the work. Given the frequency with which we wanted to change what we wrote and planned, a timebox of pre-planned work would not have worked for our writing. Using a flow-based approach to this project allowed us to deliver smaller sections and replan more often. We did not use a regular planning cadence for chapters. We did plan and replan sections at will. That allowed us to frequently adjust at the section and chapter level.
And, because finishing the sections was so important to us, it not only felt unnatural not to write every day, we didn’t need to section-level plan even once every week or two. This is similar to individual stories for an agile project.
We replanned at a higher level when we started a new chapter and that planning only required a few minutes and not an entire session. This is similar planning to an entire feature set in a roadmap.
Because we didn’t have a regular planning cadence, we didn’t have the “grind-to-a-halt” problem some teams encounter when they plan in a two-week cadence. We always had an idea of where we wanted to write next, and enough of a frame or map to start. (See [4] for how to use continual planning for your projects.)
In agile project terms, we used lean principles and continuous flow instead of timeboxes to define the work. You might think of this as the difference between a defined project framework and a lean approach to a project. We already had an idea of what to cover. We could refine our requirements and our plan as we wrote.
We found that the writing was non-linear. Writing, as is software development, is an act of discovery and learning. As we discovered ideas in later chapters, we rewrote/clarified/added to earlier chapters to establish a foundation for the new idea. Other times, we might expand a section of a chapter, realize the chapter was now too large, and split the chapter into more readable sections.
The book frame kept our project vision fresh in our minds, so we had the big picture. The story frames helped us keep a particular theme in mind as we wrote a chapter. Because we focused on the writing, we could deliver a chapter every week or two. Then, we could replan where to write next and, if necessary, what to write. As a result, we were able to complete and publish six chapters as early release seven months after we first started the project.
7.5 Pairing at a Distance
We learned quickly how to pair at distance. We discovered problems (discussed in section 3.1) and solved them. Through the solving and accommodations we made for each other’s preferences, we learned that distance pairing was just as easy as collocated pairing. It might even be easier because each person can use their preferred keyboard and mouse.
Our writing was stronger because we paired. We had the benefit of constant review and revision, which allowed us to progress quite fast on the book-writing.
8. What Else We Learned
Consider pair-writing as a simulation—an agile game—for your team’s product development. Teams that have trouble collaborating on the product might make progress with writing because it’s a low-cost experiment for collaboration. The writing might break the logjam for the team’s product development. It may also inspire your fellow developers, testers, product owners and other team members on how they can apply agile principles and practices in a distributed environment.
We would pair-write again for several reasons: we had fun, we produced a better book than we could have solo, and we discovered concepts together that we would not have realized alone. We found our short timeboxes invaluable in helping us focus and finish.
Johanna wishes for a Markdown tool that allowed pair-writing. We could not refer to locations in the book or the real images when we exported into Markdown.
Mark wishes he and Johanna could have experimented more with pairing multiple times a day. Could we produce a chapter even faster? Could we produce a book in a month or even sooner? It would be exciting to try.
In terms of distributed agile teams, we didn’t change our perspectives or value. We still believe distributed and dispersed teams need enough hours of overlap to succeed. We still believe in the necessary richness and naturalness of communication channels. We did modify our thinking by creating principles to help teams with fewer hours of overlap be more successful with the way we explained how to create more hours of overlap and the principles we’ve seen that make distributed and dispersed agile teams succeed.
9. Acknowledgements
We thank David Grabel, our shepherd, for his insightful comments on our interim drafts. We thank Rebecca Wirfs-Brock for her help in formatting the final paper.
Mark thanks his family for tolerating the many writing sessions and thanks his wife for allowing the camera to be on their shared office, perhaps too often. I did have “warning lights”.
Mark also thanks Mike Hansen, Senior Vice President of Product at Sonatype for his flexibility and support in writing the book and this paper.
REFERENCES
- Ambler, Scott. Distributed Agile Survey, http://www.disciplinedagiledelivery.com/geographically-distributed-agile-2016/
- Rothman, Johanna and Mark C. Kilby, From Chaos to Successful Distributed Agile Teams: Collaborate to Deliver, Practical Ink, 2018.
- Rothman, Johanna. Create Your Successful Agile Project: Collaborate, Measure, Estimate, Deliver. Pragmatic Bookshelf, 2017.
- Rothman, Johanna,Lean Roadmapping series,
