The authors recount the journey of a Fortune 200 utility’s path to agility leveraging the power of lean. We tell the story of American Electric Power’s agile adoption in IT, and how leveraging lean principles and a lean management system created alignment within our management ranks to fuel our current agile transformation. We demonstrate where agile and lean fit well together, and explore the challenges we encountered of reconciling agile and lean values and practices within an entrenched and long-standing traditional corporate culture. This experience report provides insights that others may leverage as they continue to move their organization toward a more contemporary workplace with greater levels of agility
1. INTRODUCTION
Hello. Welcome to our story. Joe and Greg have a combined 15 years working on the agile and lean transformation at American Electric Power (AEP). Joe’s story begins as an experienced Agile Coach and Scrum Master working with teams delivering software. Greg’s story beings as a member of the Information Technology (IT) PMO working to dismantle prescriptive waterfall. Our paths converged into a common effort to transform AEP Information Technology into an agile software delivery organization.
Our story describes how we leveraged an organizational lean transformation to establish agility as our lean software delivery method. We explain how partnering with our lean transformation counterparts was, and continues to be, crucial to our success. And we will demonstrate how we use vocabulary to continually promote agility and get our messages across to senior and executive management. Throughout the story, we will describe the successes, learnings, and difficulties we encountered, and with respect to those difficulties, how we navigated them to produce a desirable outcome.
2. Background
American Electric Power (AEP) is one of the largest electric utilities in the U.S., serving nearly 5.4 million customers in 11 states. We have 18,000 employees, approximately 1,100 of those are in the IT space, and 500 are in the IT Business Applications space. The IT Business Applications area is where our story takes place.
In the decade prior to 2010 AEP had established a heavily prescriptive waterfall practice. Projects, with heavy focus on the Project Manager (PM), were expected to follow the prescriptive methodology. Deviations from the methodology required approval from a waiver board. This resulted in unnecessary work being performed. For example, the methodology required the completion of a user interface design document. On a project modifying backend systems without a user interface, the PM had to either create a useless design document or go before the waiver board and get a waiver. The least painful path was often to create the design document containing little to no helpful information.
Teams were loose associations of people with specialized roles in a matrixed organization. The organizational matrix had already begun to be broken down as our story begins, but much of the culture still centered around specialized roles and people working on many projects simultaneously, executing their individual tasks on project schedules.
The primary measures of project success were actual cost vs. budget, meeting project schedule dates, and adherence to the methodology. This resulted in contrasting views of project success. IT could see the project as highly successful, while the business partner receiving the project deliverables could be highly dissatisfied with the results. From the IT view, the project was on budget and on schedule, which were our primary measurements of success. However from the business partners, other factors were also important to them that we did not measure or hold in high regard. These factors included the functionality and usability of the features delivered, the time it took to get the project delivered from the inception of their idea, and their experience working with the IT staff.
Override processes were established to mark projects as failed regardless of the IT measures if the business partner was sufficiently dissatisfied and took the necessary steps to indicate their displeasure. As you might expect, this created tension between IT and our Business Partners.
Pockets of IT professionals were dissatisfied with the methodology and the results it produced. Experimentation with agile practices began to emerge independently within a few teams. This story begins as the first team began experimenting with agile practices. It continues as Greg joined the IT PMO in 2010 with a mission to dismantle the methodology, and with Joe in 2011 as he joined AEP as an experienced agilist. We didn’t know one another, and unbeknownst to each of us, we were on a similar mission.
Figure 1. AEP Lean Agile Journey
3. Our Story
“That won’t work here!” The year was 2009. One brave manager in IT Business Applications (we will call him Jim) was attempting to start up the first officially recognized agile team. From his research, Jim believed adopting agile principles and practices could result in happier and more engaged teams, and consequently improve Business Partner satisfaction. Jim was joined by another IT Manager who had been researching and exploring agile methods, and a Business Partner Manager who was willing to try an experiment with agility on one of her projects. The reaction from senior management, both IT and business, was skeptical. However, to their credit, management permitted the first official agile pilot team to be created.
The team was creating a technology solution to send communications directly to customers. The result was a resounding success. Positive feedback was garnered from all quarters. It was an impressive result. Three additional agile project teams were formed, and they too had great success. The people on the agile teams were engaged and energized, qualities that could be experienced by walking in the team spaces. Business partner satisfaction was incredibly high compared to waterfall projects. However, agility was not penetrating the overwhelming waterfall culture. Success didn’t have much impact on the organizational structure. In fact, we were stuck. We had four teams using agile practices to deliver their projects, and no one else was joining the movement to agility.
3.1 Exploration and Enablement
At this time in our agile journey, we identified some key challenges. Staff, including management, were uncertain of agility and how it impacted their jobs. Management was still skeptical of agility and its applicability to our environment. The CIO was mostly unaware of agility. Front line and middle/senior management recognized the business satisfaction improvements, but felt agility was appropriate for only small, low risk efforts.
We also had another challenge created by our experiments with agility on those first pilot teams. We used contract staff to bring in agile expertise on those first teams. Each brought in their own version of agility. Some used an established agile framework, others used homegrown approaches. This caused discord within the small agile community we had built, raising questions around our agile practices. Are we sprinting, iterating, or flowing work? Are we conducting sprint zeroes, planning iterations, or neither? What agile practices are the right practices to adopt?
To expand agility, we had to establish consistency in our practices and methods. It was 2011 and we hadn’t made the progress we wanted. We needed to focus on consistency within our agile practice. We couldn’t effectively train staff and earn the trust of management while our teams were split on our practices. Given these challenges, we chose our next steps to move our organization forward. We standardized our agile practice by selecting Scrum as our framework. We partnered with a local firm that had extensive agile experience and a close relationship with Scrum.org. We established a training protocol using Scrum.org training and certification. And we established a fit assessment to select the best projects for Scrum vs. waterfall.
With those foundational decisions made, our agile practice expanded. By adding Scrum as an approved delivery framework, we made it available for teams to choose. This was our “Enablement” period. We enabled the teams to choose Scrum, yet there was not a directive to do so, nor any vision of agility at AEP.
While it was an improvement, management was still skeptical. Scrum was limited to mostly small, low risk efforts. We chose to carry on and win over management by consistently delivering quality solutions at high satisfaction. However, once again we found ourselves stuck with only a small piece of the application delivery portfolio delivered using the Scrum framework and agile principles.
3.2 Transformation
2013 began with a new corporate focus on lean. Adopting lean concepts and methods was now a corporate goal. This included all aspects of business operation and our 18,000 employees. IT was one of the first organizations to undergo lean transformation. We eagerly engaged the lean consultants. We used this platform to promote agility as our lean software delivery approach, and it worked. The lean transformation groups were focused on streamlining operations. Having a ready-made software delivery mechanism was an easy sell. Tying the IT agile transformation with the corporate lean transformation elevated the status of agility in the organization and allowed us to speak with at least an equal voice within the waterfall culture. We had the attention of the CIO!
Agility expanded with great success. Over the next few years we focused transforming our organization to agility. Joe and Greg, with others, had joined efforts to move the agile transformation forward with a single voice. We evaluated every new project for its suitability to Scrum.
An “agile fit assessment” process was established to facilitate the evaluation. Among the dimensions evaluated included the existence of a suitable Product Owner, identification of training needs, and the availability of a team room. We, to the extent possible, tried to set each project up for success. As a result of our efforts, approximately a third of our projects began leveraging Scrum as waterfall projects ended and new projects were started. However, we still hadn’t solved all of our initial problems. Management was still skeptical. The CIO was aware of agility, but was not openly supportive. Scrum was still viewed as only appropriate for small, low risk projects. Early adopters were doing well, but staff unfamiliar with agility remained wary of agility and how it impacted their jobs. Many felt that could “wait out” agility and it would go away – like many things before. We were still relying on contract staff for agile expertise and had not built a learning nor career path for the roles needed within our Scrum framework. We had plateaued and struggled to transform the organization beyond a third of the work. Agility had expanded to approximately 15 teams, around 25% of the Business Applications teams.
3.3 Expansion
Concurrent to our focus on our agile transformation efforts within the IT Business Applications space, the initial lean wave had completed its deployment across the enterprise. 2016 brought on the second phase of our corporate lean transformation. AEP engaged with a different consulting firm to refocus and reinvigorate our corporate lean efforts by introducing a lean management system and standard work practices.
IT was once again on the forefront of our lean transformation, as our CIO volunteered IT to be included in this new wave of lean practices. Because our agile growth had somewhat stalled, and due to the requirements and terminology corporate-wide of this lean management system, we could no longer rely on the “agile is lean” mantra we had used in 2013. We had to learn a new vocabulary and a new mindset in order to advance our transformation.
During this engagement with the lean consultants, we learned many lean practices and techniques that could we found could be interwoven with our agile practices. These included Value Stream Analysis, the Lean Management System, Kaizen, Visual Process Performance (VPP), Visual Process Adherence (VPA), A3 problem solving, Standard Work, Leader Standard Work, and Gemba.
Between the lean and agile consultants, there was a clash of vocabulary, ideas and mindsets. It was critical for us to adapt if agility as we desired was to survive. We (both Greg and Joe) were deeply involved with the formation of the IT value stream and the creation of IT standard work and visual management. We can’t go into a deep description of all aspects of a lean management system in this document, but in a nutshell the core is that there is standard work. Standard work is the best way we know how to do things today and everyone follows it. The organization uses visual management in the form of Visual Process Adherence (VPA) and Visual Process Performance (VPP) to indicate adherence to the standard work and if the standard work is producing the expected results. Kaizen (continuous improvement) and A3 (problem solving) are performed to improve standard work and solve problems when expected results aren’t realized.
As you might imagine, there was a clash with our agile mindset that told us that each team is unique. That each product is unique. That each effort is unique and generally not repeatable. That teams self-organize around their work and perform their own continuous improvement to optimize their delivery of value. Now every team must do the same thing? And if teams plan effectively up front that we can monitor their adherence to plan and get them back on plan? Are we starting up a prescriptive agile practice? We were seeing spots!
Although agile has its roots in lean, we found a divergence of practices and implementations that created tension and conflict. In our case, the lean consultants wanted to see standard work implemented at the team level, creating visual process adherence and visual process performance measures for management. We resisted the notion of penetrating teams with standard work to protect the desire for self-organization and innovation. We executed Kaizen events staffed by team members to work through the differences and arrive at workable solutions, and those solutions were then piloted by a subset of chosen teams before they were proliferated across all teams.
The lean consultants wanted us to visualize the Scrum Team’s plans, and their progress along those plans (which took the form of Release Burn Up charts). They also wanted to highlight the abnormal condition when the team was no longer ‘on plan’, and create a discussion with management on how they would get back on plan. We immediately recognized this as a potentially harmful activity to our agile culture, and given our history of “control” on projects, we were pretty confident that this was not the right option to further our agile mindset.
During one collaboration session with the lean consultants, they asked the question “how do you know when you need a heroic sprint?” At first we didn’t know how to respond. After all, what is a heroic sprint? Our lean consultants explained that it is a sprint to get back on plan. That sounded like a death march to us, and could possibly conflict with the agile principle of sustainable pace. With this definition, what would prevent every Sprint from becoming a heroic Sprint?
We explained that with Scrum teams, the plan emerges and changes based on emergent factors in and around the team. The burn charts still conveyed the necessary information to generate good discussion without plan lines. Sustainable pace was foundational to a healthy team so the team has the capacity to take on a rare “heroic” sprint when it is absolutely necessary. These discussions created some of the highest tension and misunderstanding between the agile and lean experts, and we needed significant discussions and shared understanding to come to resolution.
Our burn charts were introduced to the teams without plan lines drawn where actual data for the team existed (see Figure 2). We did learn the value of consistency in our visual management. Lean taught us there is a subset of information radiators that, if standard across teams, could enhance management’s ability to understand what was happening in the team rooms, and provide support as needed. We adopted standards for the most common burn charts and indeed found it improved communication with management as well as stakeholders. Occasionally our agile coaches and Scrum Team members have questions about the consistent visuals across teams and their value proposition. We have explained that they are part of our Lean Management System and provide value to the stakeholders and organization as a whole. Having the right level of consistency across the teams is part of the cost of operating in a large enterprise. And we want to continually inspect and adapt these visuals to ensure they provide the right amount of value for their cost.
Figure 2. AEP Scrum Team Release Burn Up Example
While lean and agile share many principles and have many similarities, the different language between lean and Scrum began to create confusion and discord among our employees. For example, lean introduced us to the concept of Huddles and Gemba walks. The IT group had begun to use Huddles and Gembas across the teams to review their visual management boards. In the Scrum Teams, we conduct Sprint Reviews and Sprint Retrospectives as continuous improvement mechanisms. Confusion arose in both the terminology used and the purpose of all these events. We found that using common language made agility more accessible to management and business partners across the enterprise. Since the agile transformation was being mostly led from the IT group, other areas did not know or understand the language or practices of Scrum. However, when we began to speak in terms being used by our enterprise lean initiative, the management and business partners more readily understood our new way of working. It helped create shared understanding between the Scrum Teams and other parts of the organization.
Our solution to the differences in practices and language was to partner with our lean counterparts and try to understand their perspective. Together we identified similarities between the agile and lean disciplines and built on areas where we agreed. We eliminated duplicate practices where they existed.
We also created a translation between agile vocabulary and lean vocabulary. We learned to use the right vocabulary with the right audience to avoid misunderstandings and confrontation when we were in agreement. Our goal was not to teach the organization new vocabulary. This translation was mainly used by us and our lean counterparts to communicate more effectively with other parts of the organization horizontally and vertically. If we were speaking to people familiar with agility, we could use the agile terms. When we spoke to the broader organization and senior leaders, we used lean terms.
We worked with our lean counterparts to achieve the organizational consistency needed while maintaining the agile teams’ ability to self-organize around their work. This solution required a delicate balance, and we did hear concerns from both disciplines. The Scrum Teams felt that they were getting too much prescription placed on them, and some of it was not valuable. Our lean partners felt there was too much variability across the teams, which made it difficult to implement a lean management system and provide consistent visuals to the organization. Our solution, which we are continually inspecting and adapting, was to follow the Scrum framework at the team level, and have the teams use a few standard templates for their internal information radiators. To satisfy the organizational needs, the teams also have a few standard visuals to use when communicating outside the team to stakeholders and the organization at large. Everything else they do is is created from their self-organization and continuous improvement ideas.
The solution, while seemingly easy, was quite difficult. We were successful though, and we are still growing and improving our combined agile and lean practice. The results, while not perfect, have enabled us to overcome the remaining problems that have existed since the beginning. Management is no longer skeptical of agility. They have established an aspirational goal of 100% agility in the Business Applications space that we are working toward today. This goal was established to support our lean goals of keeping customer satisfaction at high levels and reducing the time it takes a Business Partner’s idea to go from inception to production deployment.
While there are still pockets of staff reluctant to both lean and agile, they understand where the future is going and are adapting. They are seeking out coaching and training to the extent we struggle to keep up – a good problem to have.
We are still relying on contract staff for most of our Scrum Masters and coaches, but we have begun building internal talent. We have more employee Scrum Masters and coaches than ever before and we look forward to expanding the ranks. We have built learning paths for Product Owners, Scrum Masters, Development Team members, and management that they may leverage to help them be successful in their transition to agility.
We have also gained some unexpected benefits in partnering with our lean counterparts. Lean, through Gemba, gets management into the team room where they can inspect the work where it is occurring, get to know the teams, and form more collaborative working relationships. We are actively coaching management on reading and interpreting information radiators, which is made easier by having standards around information radiators most commonly consumed by management. Lean, through Kaizen and A3, provides us a way to help reduce organizational impediments that were nearly impossible to address in the past. Lean has elevated agility to the highest levels of the company, and those executives are now interested in our agile practice. They occasionally visit our agile rooms where in the past it was extremely rare to see them in the IT spaces. We have established a continuous improvement cycle with the standard work. This allows us to modify and improve standard work to enhance effectiveness and eliminate pain points, and to evolve it as business conditions change.
4. What We Learned
Over the course of the past nine years, we’ve learned valuable lessons from our experience of adopting agility and lean, blending them together into a cohesive movement. And in true agile and lean fashion, we are still learning as we progress on this journey.
We learned that lean principles and practices can be leveraged, especially at the management levels, to enhance and accelerate our organization’s move to greater agility. With our corporate lean initiative being supported by our executives, it introduced lean concepts across the organization. Since agile is rooted in lean principles, we could speak to leaders in lean terms regarding what we were doing in the IT space to focus on the top priorities, reduce WIP, and continuously improve.
We discovered that in a mature enterprise like ours with traditional management mindsets and comfort executing big construction projects, lean language may be more accessible than agile terminology. AEP has nearly 18,000 employees trained in lean. Only 500 of those are in the IT Business Applications space, which is where our agile efforts are focused. Using the language of agile and Scrum to the majority of the organization could be off-putting and confusing. We often had to describe what we were talking about when using these terms, and even then, it was difficult for folks unfamiliar with agile to fully understand. When we used our lean language to describe our goals and execution practices, it brought understanding to the conversations and reduced the complexity of the conversations. Aligning agile values and principles with lean language helped eliminate waste, enhance value delivery, and improve speed to market across IT teams.
As mentioned previously, using lean terminology with people outside our IT group eliminated confusion and lack of understanding about what we were doing in our move towards greater agility. We were able to speak directly about how our practices supported reducing or eliminating the lean wastes, maximize delivery by minimizing work in progress, and deliver value to production sooner and more frequently. By demonstrating that the teams were working on these goals using lean terminology and the Lean Management System, support for our efforts was expanded across and outside our IT Business Applications space. We conducted conversations and problem-solving sessions focused on opportunities uncovered through our use of lean techniques at the management levels. Since we were using language and practices familiar to everyone across AEP, there was alignment with others to solve these problems and continuously improve. Had we recognized this sooner and begun using lean terms in our conversations, we may have enjoyed earlier support from our executives and possibly others across the organization. As it was, the executives weren’t opposed to agility, they were either unaware or indifferent.
We found that combining our agile adoption with an enterprise lean initiative fueled, supported, and accelerated our move towards more agility. Instead of looking at lean and agile as two separate initiatives, we combined them into one overarching movement, with Scrum being our lean implementation in IT Business Applications. We recognized the need to do this fairly early in the initiatives, yet we could have done more socialization around this one movement at the team level to reduce questions and confusion about these two seemingly competing priorities.
Leveraging lean techniques (coaching, standard work, value stream mapping) at the management level helped us gain greater momentum and acceptance in our agile adoption and continuous improvement efforts. Once we reached approximately 15 Scrum teams, we stagnated in our expansion of agility. The early adopters and those who became excited about agility had opted in, and everyone else continued to do what they had always done. Traditional management and waterfall delivery was still the norm. Once we engaged the second lean initiative, created our Demand to Delivery (D2D) pipeline value stream, and committed to improving our speed to market, we experienced a large expansion of agility across some of these traditional areas. The D2D effort introduced us to the Lean Management System, VPP, VPA, and standard work. As a result, we grew to approximately 35 Scrum teams within two years, where earlier it had taken us seven years to get from zero to 15 teams. It also led to us setting the aspirational goal of 100% agility across all Business Applications teams.
We were, and still are, continuously reminded that changing management mindsets and behaviors is very difficult and complex, especially if management does not change. People value safety, and with safety being AEP’s number one priority, the idea of safety permeates every facet of the organization. This includes people’s safety about their job and status in the organization. We spent a great deal of time focused on the teams and helping them move towards greater agility. We also spent some time with their immediate management to mentor them on how to support this new team model. Unfortunately, the front line management’s leadership was asking them for the same things they always had in the same format. And this legacy management style was very hierarchical, utilizing a traditional command and control mindset. With their management asking for the same old things, it sometimes didn’t feel safe for the front line management to do something different than what was being asked, or even suggest a different way of doing things. This hurt our ability to create a safe environment where the individuals on the teams could run experiments, suggest new ways of working, and take ownership of making change happen. There is an aversion to being vulnerable. We may have enjoyed greater success in cultivating the culture we wanted if we had one or two senior leaders who employed more of a servant leadership style, held to that style under pressure, made their behaviors and the reason for them big and visible, and fully supported their parts of the organization in displaying those same behaviors.
We learned that Lean Leader Behavior coaching can help enable the emergence of self-organization and accountability on the teams. The implementation of the Lean Management System included the practice of Lean Leader Behavior coaching. This introduced a systematic way to coach and mentor management on the team outputs that are important to management and stakeholders of each team. It also reinforced the behaviors and techniques management needed to adopt to enjoy greater success from their teams. This applied specifically to accountability and self-organization, which were expected to lead to better results in delivery from the teams. Our lean coaches had a solid plan for conducting the coaching at regular intervals and providing feedback loops to help the management improve in their interactions. As with any large number of people, results in the effectiveness of the coaching have been mixed across the management staff, and people are in widely different places in this journey. It has created team cultures that vary across our organization. This is an area that we’ve struggled in to determine how to be more effective. Perhaps if one of the senior leaders had emphasized this practice more strongly, demonstrated it visibly, and held the management staff accountable to it, we would have better results. The danger is that we could create the perception of command and controlling our move towards less command and control behavior.
5. Epilogue
While working to proliferate agility across the IT Business Applications area, we were excited for the corporate lean initiative as we both believed it would help support our move to agility. What we did not anticipate is the extent to which the lean initiative positively impacted our agility, especially once we combined the two into one movement. We also did not foresee the friction and discord realized with the implementation of lean practices in and around the agile teams. This is something we are still balancing, as the organization pushes for standardization and the teams embrace self-organization and the relentless pursuit of value delivery.
As our lean and agile practices continue to expand across the rest of IT Business Applications and beyond, we are excited to see what new challenges arise. We hope that our experience helps others along their journey, especially where lean thinking and practices are embedded in their organizations. Using lean language and techniques such as Kaizen, standard work, PDCA cycles, and Gembas within your agile practice is more familiar and inviting to executives and those outside the agile teams, which may help you gain the support you need to continue to move agility forward.
6. Acknowledgements
We acknowledge James Brosnahan, AEP IT Continuous Improvement Manager, and Chris Miller, AEP IT Demand Director, for their courage and vision that agility would work at AEP and provide tangible benefits to our customers. We also acknowledge Kathy Kelley, Customer Interface & Channel Management Manager, who was the first AEP Business Partner to experiment with agility, and later pronounced her desire to adopt agile practices for every future initiative. Our thanks go to Bruce Pecci, IT Director, for his ongoing support of a more humane, accountable, and engaged working environment. We also thank Dick Mills, IT Manager, for his support in our desire to continue to learn from and share with the agile community, and his focus on bringing in people and tools to support our adoption of technical practices.
We want to extend our appreciation to the employees of AEP who believe that adopting the agile values and principles will create a better world of work. We also appreciate our colleagues in the worldwide agile community, who keep us inspired, curious, and challenged with being our best.
And last but certainly not least, we thank Rebecca Wirfs-Brock, our shepherd from Agile2018, who helped us corral our thoughts and stories, and bring clarity into this Experience Report. Her guidance helped us create an artifact that we believe is much more valuable to our community thanks to her input and engagement with us.
REFERENCES
Brosnahan, James, “End of Year Summary, Agile Enablement” December 2012
Astolfi, Joe and Goodwin, Josh, “When Lean and Agile Converge: The I.T. Transformation at American Electric Power”, June 2018
