This report covers my time as an Agile Coach at Toyota, from August 2017 thru May 2018 including my attempts to build high-performing teams thru techniques such as Crew Resource Management and Red Teaming (Critical Thinking) and the challenges I faced.
1. INTRODUCTION
While Toyota is known for the Lean practices developed as part of the Toyota Production System, their prowess on the production line didn’t equate to a well-running Information Systems (IS) department. An Agile transformation began in 2016 to move the North America IS group from waterfall practices to Scrum. This report is based on the experiences of myself as one of the coaches working on that transformation, the challenges I faced, and the tools I used to move the transformation in the right direction. The discussion will include how concepts such as Crew Resource Management (CRM), High Reliability Organizations (HROs), Red Teaming, and the ZoneFive tool from AGLX were used to build high-performing teams.
CRM was first developed as a response to a number of airplane crashes in the 1970s such as United Airlines flight 173, which ran out of fuel and crashed short of the runway while the crew was troubleshooting an issue with the landing gear indicator. More recently, CRM is cited as one of the reasons there were no causalities in the 2009 crash of US Air flight 1549. The elements of CRM are often referred to by the acronym DAMCLAS, which stands for Decision Making, Assertiveness, Mission Analysis, Communications, Leadership, Adaptability, and Situational Analysis. Some examples of how CRM applies to the agile coaching field can be found in section two.
HROs developed out of complex, highly volatile environments such as aircraft carriers and nuclear reactors that are able to avoid catastrophes in spite of their volatility. Red Teaming originated in the military as a way to think from the enemies perspective. Again, examples of how these apply to coaching will be found in section 2.
The discussion will also talk about how the Scrum@Scale model was applied and the challenges in getting managers and executives engaged as chief product owners and Metascrum participants.
2. Background
I have been practicing agile for over ten years, most of that time at IBM. My typical engagement at IBM combined delivery of software applications developed by a team of IBM developers and educating the client on our scrum-based methodology. I did receive formal agile coach training and had a few engagements that were coaching specific before leaving IBM for Toyota.
During the first couple months at Toyota, I had the privilege of working with a couple coaches from Scrum.inc before their contract expired. Then I was on my own, coaching as many as 19 teams in two locations; Georgetown KY and Ann Arbor MI. With this many teams and having to travel at least once a month, I felt from the beginning I never really got to know all my teams well. I was at Toyota for ten months before my contract ended.
I should also mention my professional career began as a cryptologist in the US Navy, including time on an aircraft carrier and work on the U-2 program/Lockheed Skunkworks.
Ten months before I started, all the teams in my department went from waterfall the Scrum. They received the standard CSM class and then a couple weeks of close coaching as they made the shift. They developed team operating agreements, definitions of ready and done, put together a backlog, and then planned their first sprint. They continued to get coaching for the next ten months, but not as focused. Some teams caught on and were following Scrum pretty well and showed they understood the principles behind the practices. Other teams struggled and in some cases were not even performing all the Scrum practices by the time I arrived.
3. The Story
3.1 Situational Awareness and The Scrum Master Having Trouble Counting
It was the end of a team’s 5th sprint. This was a team I helped from their launch and they were starting to come together. The Scrum Master was excited because the team had finished 45 story points, their most in a sprint. I arrived at the demo and the Product Owner reported only 32 story points were completed, below the sprint goal. What went wrong?
The Scrum Master lost situational awareness, which is defined as how well a person’s view of the situation matches reality. In this case, it didn’t match very well The Scrum Master was so focused on the number of story points completed, he lost site of the real definition of done. This is a phenomena called channelized attention and it’s been the cause of airplane crashes.
The team discussed this during the retrospective. They came up with a couple kaizens. First, everyone was going to review the definition of done together during sprint planning. Next, they were going to post the definition on their team board to make it visible.
One area of focus within CRM is human factors; which looks at how people interact with their environment and designing the system to optimize this interaction. One way to do this is through visual cues. A phrase often heard around Toyota was “Make it visible.” One way to do this was by creating physical card boards for the teams, even though tools such as Jira were also in place. In this case, the team had a board for tracking each sprint. They improved this board by adding the definition of done.
With a good board set up, there were no longer questions about how many points were done at any time during the sprint., any impediments were in clear view and the team didn’t lose track of their definition of done.
3.2 Red Teaming and getting the Sprint Plan right
The Applied Critical Thinking Handbook (formerly the Red Team Handbook) defines Red Teaming as a “capability to fully explore alternatives in plans…” The Handbook talks about the importance of Critical Thinking and how this skill set must be taught. In planning, group cohesion can lead the team to conformity and group think (p.93). The red team perspective is to get the team to think about what could potentially occur.
One of my teams was wrapping up Sprint planning. The product owner had prioritized a set of stories that all met the definition of ready. The team had determined what their capacity would be for the Sprint and picked stories from the prioritized list until they hit their capacity. The team collectively came up with a sprint goal and they were ready to go.
At this point I posed the question “what could go wrong with this plan” to get them to think critically. At first they were silent. Then one of the developers spoke up. He pointed out one of the stories they had planned for later in the Sprint may have some technical challenges. He suggested moving it to the first story they worked on. One of the other developers brought up a similar suggestion with another story. The team re-arranged the stories in their Sprint backlog and now felt more confident about the order in which they were going to do the stories.
3.3 Building Human Factors
As mentioned above, the idea of making things visible was part of many conversations. One tool that existed within Toyota was circle, triangle, cross, as in Figure 1.
Figure 1. Circle/Triangle/Cross
The Circle means everything is going well, the triangle means some issues that may need attention, and the cross means major issues. This visual was used in a number of areas. I used it as a quick visual on team maturity that I could share with leadership. They could easily see the teams that were performing well, those that needed a bit of coaching, and those that needed much more attention.
3.4 Are we Safe?
One of the activities I was assigned at the start of my engagement was to collect data related to an impediment. The impediment was that people were afraid to speak up; i.e., a lack of psychological safety. I interviewed all the Team Members in the organization except the senior leadership and found that 32% did have at least occasional fear to speak up.
Figure 2. Results of “Fear to Speak Up” Impediment
I presented this information plus proposed countermeasures to the leadership team…and nothing happened. They took no actions. I had heard senior leaders state “people aren’t afraid to come to me” but the empirical evidence said otherwise. This survey didn’t even include the contractors. I can say based on observations that the data would be even more skewed towards a safety issue had the contractors been included in the data collection.
3.5 Fighting Group Think
Any time you bring a group of people together, you can get group think. One person makes a comment and everyone falls in line with that idea rather than expressing their own. There can be a number of reasons for this; deferring to the senior person, lack of psychological safety, or even just being a bunch of introverts.
There are ways to prevent group think. Planning poker was designed so that group think doesn’t happen; that is, when it is done properly. However, if the Scrum Master asks everyone what their estimate is, you will probably get group think. This is a pattern I had to address on several teams.
Retrospectives is another area where group think can dominate. To address it here, I used a technique called Think-Write-Share, which can be found in the Critical Thinking Handbook. Imagine running a simple retrospective where you have two questions; what went well and what do we need to improve on? You ask everyone to think about their answers in silence. Then you have them write down ideas on Post-It Notes. When it seems everyone is done (or timebox the activity), you have them share their ideas by putting them up on the wall. Then the conversation starts and you have them do some affinity grouping and dot voting to move the retrospective along. Using this approach lets every voice be heard. The first person to speak doesn’t become the only idea that’s discussed.
Group Think can also take hold when one person dominates a conversation. I was observing a “wall chart” meeting; essentially a status meeting for the Chief Product Owner (CPO). One of the four product owners began their update. Questions came up from the CPO and before anyone realized, 35 of the 60 minutes allocated to the meeting were used up. The second product owner used most of the remaining time, leaving no time for the other two product owners to provide any meaningful information.
I introduced the group to the “Circle of Voices” technique (also in the Critical Thinking Handbook). Each product owner would be given 5 minutes to talk uninterrupted. If anyone had questions, they would write them down. After everyone had their first chance to talk, the CPO went back around asking questions. This gave everyone a chance to be heard and the CPO then had a better picture across all the work so they could focus their questions on the most important items.
3.6 How to Communicate
Assertiveness, combined with communications, are key in CRM. If a co-pilot or nurse is aware of something wrong, they should not hesitate to tell the pilot or doctor. The same applies if a developer wants to bring something up to the Product Owner.
I was in with one of my teams during the stand-up. The product owner wasn’t there on that day. A developer made a comment about being hesitant to tell the product owner something. I introduced her and the team to a technique called SBAR-Situation/Background/Assessment/ Recommendation. It is used often in a hospital setting when someone needs to bring something to the attention to the doctor and is an example of closed-loop communications. I walked the developer through how should could use it to effectively bring a recommendation to the Product Owner (in this case, Bill). The conversation could go something like this:
[Situation] “Bill, we’re having trouble with our validations. [Background] We set up using server-side validations because we thought they would be easier to maintain. [Assessment] The time to run validations is very long due to latency issues. [Recommendation] I suggest we use client-side validations to improve performance. Can we add that to the backlog for the next sprint?”
The developer addressed the Product Owner by name to ensure she had his attention and then quickly walked through the SBAR dialog, ending with a question requiring the Product Owner’s response.
3.7 The Collective Mind
While most of us don’t work in an environment along the lines of High Reliability Organizations (HROs), we can still take some of their concepts. Two in particular are the drive for continuous improvement and the development of collective mindfulness.
The book Team of Teams deals with the US Army in Iraq. It describes how the organization evolved into a team of teams to meet their threat more effectively. This idea is also reflected in the Scrum at Scale guide.
This approach was being adopted at Toyota. Metascrums were being held to bring together the product owners under a CPO. However, I didn’t see collective mindfulness evolving. One challenge, everyone brought their computer to meetings and when they weren’t talking, they were nose deep into something else and not paying attention. I introduced the idea of collective mindfulness and suggested that meetings be held without computers. As the meetings evolved, they became a group of people standing together around a wall chart and not people sitting around a table with their computers.
3.8 Measuring Things
Throughout the time I was at Toyota, I was using a tool called ZoneFive by AGLX (see appendix). It provided me a way of evaluating where each team was across 6 areas; Backlog Refinement, Planning, Stand-Ups, Teaming, Reviews, and Retrospectives. There were also 2 categories (Leadership & Organizational Culture) where the team could provide their view of the organization. For each area, the team self-scored on a 1-5 scale. Narrative descriptions of levels 1, 3, and 5 were provided. I also scored the teams. The full tool can be found in the appendix.
When I first started, the scores for my assessments, averaged across all teams. I re-assessed the teams six months later, in early 2018. Figure 2 shows the results of the two assessments.
Figure 3 – Maturity Assessment Results
As the data shows, the two biggest improvements came in the daily Scrum and Retrospectives. Backlog refinement actually got worse and planning was effectively flat. There was some improvement in running reviews.
What does this data show? Some of these techniques did improve practices, but there was not an overwhelming improvement across the board.
These tools did help in some areas. Tools such as think-write-share had a big impact on retrospectives. This tool removed group think and resulted in more productive retros.
Why was there no progress in backlog refinement or planning? Both of these activities are heavily dependent on the product owner. I mentioned above that the product owners were Team Members while the rest of the teams were contractors. I also mentioned the psychological safety issue that existed. The net impact of these facts was that the Scrum Masters did not feel safe coaching the product owners on their role, but even that is an over simplification of bigger issues.
One of my last activities before my contract ended was conducting workshops with the product owners and scrum masters with the goal of identifying challenges to the Scrum implementation and associated root causes. A couple themes surfaced; the product owners needed better training, they felt there wasn’t time to do the product owner role, they needed more people, they needed less work and priorities were still … Toyota had a hiring freeze and couldn’t bring in more people to be product owners.
3.9 Self-Awareness
The Critical thinking handbook dedicates a chapter to self-awareness, stating that “The self-aware person is more enabled as a critical thinker, more aware of personal biases and recognizes his or her own cultural framework.”
One recommendation in the handbook is daily journaling. While I didn’t journal daily, I did make the effort to journal at least once per week, usually at the end of the week. I have been journaling off and on for about 15 years, so this wasn’t a new practice. One way I used journaling was to capture my accomplishments for the week. That also helped me focus on what I would work on in the coming week
This was my first full-time coaching gig and when I’m in a new situation, I tend to sit back and observe rather that jumping in right away. Through my journaling I recognized situations where I should have spoken up and that helped me become more engaged.
4. What I Learned
The tools I’ve discussed are very effective in building high-performing teams. When these are added to the Scrum framework, they act as a multiplier. However, that alone won’t make a successful agile transformation.
- Being a coach for 15+ teams is too much. I should have focused on just a few teams at a time. The teams were on one week iterations, so even trying to attend all the ceremonies was impossible. At best, I could focus on three teams in a given week, and then only if their ceremonies were at different times. I had four teams end their sprint on Friday so I could only attend 2 demos/retrospectives. It would be a month or more between visits to a specific team and I felt a bit like a seagull coach…swoop in, crap on things, and fly away only to come back a month later. The teams didn’t make much progress this way either.
- Being a team coach and organizational transformational coach at the same time is too much. I should have insisted the PMO get engaged at the organizational level so I could focus on the teams. As mentioned in #1 above, I already was not giving teams enough time. Even then, I was being pulled away to help on other work such as developing a department wide communications approach, meta-scrum prioritization, or the “Fear to Speak up” survey mentioned above. I would have been more effective if I had a second coach dealing with the organizational/scaling aspects while I focused on team coaching.
- You can’t solve psychological safety by just focusing on the team and if management isn’t willing to address it, you may want to reconsider your options. In order for a team to feel safe, they need to know that it’s ok to fail or raise impediments. When management doesn’t show that it’s vulnerable, it doesn’t convey a safe environment. When scrum masters are let go and no one knows why, it will make any remaining scrum master careful about what they say. I had Scrum Masters that would share things with me and then ask for me to intervene because they didn’t feel they could tell their Product Owners the same thing. I also saw impediments stop being raised because the management team wouldn’t do anything about them. There were impediments on the Leadership Action Team board when I started that were still there when I left 10. Months later.
While some of these tools are found in a military document, they didn’t come across as command and control. Most of the teams I worked with were open to the tools and saw the benefits of them.
5. Acknowledgements
I want to thank the coaches I worked with from Scrum.inc, especially Brian “Ponch” Rivera, Dave Slaten and Nigel Thurlow. I also want to thank my shepherd, Mike Griffiths. Mike was one of the first to talk about agile at PMI (Project Management Institute) conferences and we worked together on the launch of the PMI Agile Community. We’ve also run the beaches of Florida and biked the Canadian Rockies together, Thanks, Shepherd, I couldn’t have done it without you!
REFERENCES
The Applied Critical Thinking Handbook, V8.1, found at https://usacac.army.mil/sites/default/files/documents/ufmcs/The_Applied_Critical_Thinking_Handbook_v8.1.pdf
General Stanley McChrystal, Tantum Collins, David Silverman, Chris Fussell, Team of Teams, New Rules of Engagement for a Complex World, Portfolio Publishing, 2015
