Against the backdrop of Fannie Mae’s transformation to enterprise agility, this Experience Report will focus on the case for change, Fannie Mae’s journey and the corresponding challenges, benefits and key learnings realized. Thus far, we’ve learned that while it is important to build bridges with business stakeholders, mature the capabilities of Agile teams, leverage automation as well as embrace the values and principles of the Agile manifesto, at the outset, a successful and longstanding Agile transformation is dependent upon an unrelenting focus on changing the ecosystem supporting the organization’s change itself.
1. Introduction
Over the course of the last two-and-one-half years, Fannie Mae has worked aggressively to transform itself from a siloed command and control culture, following a gated workflow with long release cycles, to an Agile organization. Today, Fannie Mae is a more dynamic value-oriented organization that is responsive to customers, focused on achieving greater efficiency by enabling fast-feedback loops, as well as uses empirical data to mature and persist Agile values and practices.
2. Background
Fannie Mae, a leading source of residential mortgage credit in the U.S. secondary market, provides reliable, largescale access to affordable mortgage credit across the country so people can buy, refinance, or rent homes. For the most part, Fannie Mae had been a pure, traditional, waterfall-based development shop with all of the inherent challenges this implementation approach had to offer. Specifically, IT and business alignment was difficult, projects were large multi-year efforts that were inherently risky and were either frequently delivered late or with reduced scope. In 2014, Fannie Mae began the journey to improve its ability to predictably deliver business capability, and to do so, with much less technical risk. Since that time, Fannie Mae has rapidly moved to an Agile product delivery model and the benefits achieved to date have exceeded our expectations. This is the story of our journey.
3. Fannie Mae’s Journey
Prior to 2014, individual teams experimented with Agile practices. As early as 2004, teams in one portfolio had implemented XP and scrum. Other efforts in different lines of business had experimented with Agile practices, but these were largely localized to small efforts and were undertaken with little, if any, organizational support. Over the years, the organization imposed traditional development and governance processes on the teams, inhibiting their ability to mature and, in most cases, causing teams to regress. At its core, Fannie Mae was wedded to a traditional phase-gate approach with long release cycles. Our budgeting, planning, PMO, Software Delivery Lifecycle (SDLC) procedures, and governance processes all drove initiatives to become large, risky projects. The result was an environment where business and IT sat on different sides of the aisle, frustrated when things inevitably went bad. Late in 2014 this began to change. Fannie Mae engaged two executives who had experience working in an Agile environment and who, together, started advocating for adopting Agile values and practices. One of these executives led the push for Agile software development, driving Agile adoption within the Enterprise Data Organization. The second executive led the Development Services (Dev Services) organization, which focused on establishing the Dev Ops platforms that would support the Agile tooling and technical practices. Importantly, both joined Fannie Mae with the mandate to drive Agile/Dev Ops practices within their organization and, if successful, to then take the mandate to the broader organization. Together they became the primary champions of Agile.
2015
Prior to 2015, Fannie Mae had been directed by its regulator to establish a Common Securitization Platform (CSP) where both Freddie Mac and Fannie Mae (and potentially other banks in the future) would issue securities. This was a highly visible and critical initiative. It was also a highly complex program. It meant two organizations had to break apart their core business processes to integrate with a third-party organization that didn’t yet exist. The result was that requirements were unknown and outside the control of Fannie Mae, yet a delivery date had been mandated. The new Agile leadership selected two critical efforts within this program, WIRE and Enterprise Data (EDI), to prove that Agile could both reduce development risk and scale effectively to deliver large, complex efforts (two other efforts were also experimenting with Agile). Both teams started their Agile transformation in January.
Initially, the two efforts focused on building their Agile capabilities within their teams. EDI focused on refining the scrum experience e.g. establishing two-week sprints, coding within a common mainline, testing in-sprint, and establishing and maintaining a product backlog. Over the first three months, new teams were stood-up and the basic Scrum practices were being adopted. In WIRE, the teams focused on building the CI/CD environment, automating test practices and establishing the sprint cadence. By the end of March, seven teams had been stood-up (6 EDI, 1 WIRE).
The first “proof” addressed risk mitigation. WIRE and EDI attempted to integrate with each other early. The integration proved very challenging with a “requirements miss” of approximately 25 percent. However, this experience was deemed a huge success. Normally integration testing would not have happened until the end of the development cycle, by which time, fixing these issues would have been exceptionally difficult, costly, and, in all likelihood, would have impacted delivery dates. From that first integration and moving forward, WIRE and EDI integrated every sprint, greatly reducing the program risk and demonstrating the ability for Agile teams to better manage risk overall. Importantly, this became the first “legend” – a story that resonates with key stakeholders and takes on a life of its own. In this case, the story of how risk was mitigated was repeated time and time again, creating momentum for the transformation.
Additionally, during this time, the Agile transformation executives worked with the key governance organization, our Project Quality Office (PQO), to update the Software Development Life Cycle (SDLC). Historically, the SDLC dictated a phased development process. It was an extremely heavy, document centric, approach to software development. While an “Agile Procedure” existed, it was Agile in name only. Based on the experiences in EDI and WIRE, the SDLC was updated to begin to reflect a true Agile development process. The focus on this effort turned out to be critical for a number of reasons. First, it removed a lot of overhead on the teams. While not perfect, it did significantly lighten the documentation overhead that had previously placed a burden on the teams. Second, and more importantly, it signaled to the entire organization that Fannie Mae was serious about the “new” methodology.
By June, EDI had 10 teams working on the EDI effort and needed a mechanism to scale. The teams had been leveraging a Scrum-of-Scrums approach to scaling, but that was proving insufficient to support the growing complexity. By this time, not only did the EDI and WIRE teams need to integrate with each other, they also needed to integrate with several traditional waterfall teams, as well as third-party organizations, to get the business capabilities required, delivered. Additionally, both teams, and management, were struggling to predictably forecast when capability was going to be ready for integration testing with enough time to then integrate with the different waterfall teams. To support the bi-model environment, the Scaled Agile Framework (SAFe) was chosen as the framework to support scaling.
The first SAFe Agile Release Train (ART) was implemented in July 2015 and the corresponding benefits were immediately apparent. Many of the internal conflicts between the teams were removed. More broadly, management now had greater visibility as to the ART’s work-in-progress as well as how that work contributed to, and aligned with, the overall CSP program. By October, the entire CSP program moved to align with the EDI Program Increment cadence. By doing so, the Agile teams (WIRE and EDI), which were striving to integrate daily, aligned with the waterfall teams to deliver working functionality to an integration environment every 10 weeks. This proved the ability to scale, as well as continued to reduce overall program risk. Additionally, in Q3, a major scope change came from the stakeholder, Common Securitization Solutions (CSS), that impacted all of the teams in the program. For traditional project teams, the work to understand the implications of the change to the CSP program was complex, taking over four-weeks to complete the impact analysis. For the Agile teams, the same effort to analyze the impact, and prioritize the changes, was completed in one afternoon. As a result, there was now a greater realization regarding the ability of Agile teams to effectively adapt to changing business requirements. With this realization, a second “legend” had been born.
By the end of 2015, the organization had 38 Agile teams and 25 percent of releases were registered as Agile. More importantly, however, was the fact that executive management had now seen the value of Agile and sought to spur further adoption by establishing the goal that 50 percent of software releases be developed by Agile teams in the next year.
2016
This year marked the rapid growth of Agile across the enterprise. A key part of this effort was the establishment of an Agile Center of Excellence (COE) which was charged with leading the Agile transformation across the enterprise. The mandate for the COE was to leverage the knowledge and experiences of the WIRE and EDI teams for the benefit of the broader Operations and Technology organization. The COE focused on creating and conducting an internal training program for management, as well as for teams, to establish a shared understanding of the Agile Manifesto and Agile software development. Over the course of 2016, over 4,900 people participated in Agile training. Simultaneously, training was reinforced with coaching. Agile coaches provided additional follow-on support to over 60 percent of the Agile teams as the COE sought to provide a consistent approach to adopting mature and persistent agile values and practices at the team and program layers of the organization. Providing experienced and skilled Agile coaches and Scrum masters was critical to supporting the transformation. In this regard, Fannie Mae relied heavily on our experienced Agile consultancy partners to both scale with us as well as to align to our Agile approach.
Over the course of the first half of 2016, the number of teams increased from 38 to 155 Scrum teams. To create this momentum, the COE focused on developing the ecosystem required to support the organization’s dynamic growth instead of attempting to govern the transformation. In other words, while the Agile COE guided, influenced and worked in support of the Agile transformation, it did not dictate it. Ownership and accountability for Agile adoption and maturity rested with each of the portfolio’s leadership who were then accountable for its success.
During this time, Dev Services initiated three tools to help guide the transformation efforts.
- Initially, an Agile Assessment tool was used to assess maturity in the adoption of Agile values, principles and practices. Since the COE allowed teams and portfolios the flexibility to manage their respective Agile transformation, we needed a way to measure progress and understand the state of Agile across Fannie Mae. To do this, the COE implemented the Agile Maturity Model (AMM).
Figure 1. Agile Maturity Model scale used to evaluate teams.
The AMM had some key attributes that the COE found necessary for it to be effective. First, it had to be easy enough to use so that it did not require an Agile Coach to administer it. Second, the attributes that made up each level had to be as clearly-defined, and as non-discretionary, as possible. Consequently, this tool primarily focused on our ability to change practices, but doesn’t really assess progress changing culture. The assessment is completed monthly by the scrum master working in collaboration with the Agile Coach (when there is one embedded in the portfolio).
- Thereafter, the Agile Health Radar tool, used in addition to the AMM, allowed the teams, and their management, to gain greater insight into the culture underlying the transformation, both with respect to their own initiatives, as well as initiatives taking place across the portfolio.
Figure 2. The Agility Health Radar provides insight into the cultural transformation of the teams along five categories: clarity, performance, leadership, culture, and foundation.
- Lastly, the CAST Application Intelligence Platform (AIP) is a static code quality management tool that was used to capture key quality and productivity metrics that provided insight into the code being produced. This tool demonstrated that Agile not only mitigated risk, but also proved to be a more productive approach that resulted in higher quality code being generated.
Figure 3. CAST analysis of Productivity and Quality Improvement for Operations & Technology.
Dev Services ran the CAST AIP tool against the code bases on six different applications. In the first analysis, the release was delivered following a traditional approach and then, as part of subsequent release, delivered following an agile approach. The tool compared the results of the last waterfall release with the first Agile release for the application. In all six cases, the team remained the same. When we ran the tool we found that the teams produced on average 30 percent more function points when using an Agile approach and the code was 50 percent better in quality. The CAST AIP tool’s results proved immensely valuable in finally putting to rest any remaining resisters.
In addition to supporting the development teams, the COE focused on working with the business to implement lean management practices. To this end, the business had engaged a strategy consultant to help the organization become more customer-centric by leveraging lean management techniques. While most of these techniques were focused on business processes/value streams, a major component centered on how Fannie Mae developed, and launched, new products. This effort, called WoW/Agile, was launched in February, as part of which, the business brought design thinking and Lean Startup concepts into the organization’s software development world. This change in approach proved pivotal in the transformation. Up until this point, Fannie Mae struggled to get consistent business engagement in agile development initiatives. On some engagements, the business was hesitant to engage. In others, the business provided the product owners. At our best, we were able to get the business to engage enough to develop the roadmaps that would guide our teams. When the business did engage, they did so to be good partners, but continued to view these initiatives as a technology effort, not core to the business. WoW/Agile introduced design thinking and Lean Startup concepts that spanned both business and technology and the business began to recognize that the agile transformation wasn’t just a technology effort. Rather, it was core to their success in achieving their objectives. In October, the business established the product development organization where design thinking and data insight are at the core of how Fannie Mae manages our product suite.
In addition to building out this capability, the WoW/Agile effort aligned the entire organization to Agile development. As a result, both business and technology began working together to solve some of the corporate challenges to Agile development. The COE focused on aligning the facilities, procurement, governance reviews, budgeting, and ePMO processes to better align to support an Agile product development model. Additionally, a significant investment was made in the tools and techniques to facilitate faster adoption of Agile within the development teams by:
- Accelerating the adoption of Dev Ops capabilities by kick-starting platform upgrades, developing initial CI/CD patterns, building out Rapid IT blueprints, and introducing Test Data Management and Delphix capabilities.
- Accelerating Test Automation activities by supplying/training testers and engineers in Test-Driven Development (TDD) and Behavior-Driven Development (BDD) methodologies for new Agile teams.
By the end of 2016, Agile had taken hold. We had achieved the 50 percent Agile release goal established by the leadership team at the end of 2015 and now Executive Management mandated that 75 percent of all releases be performed by Agile teams in 2017. Moreover, this goal became a Board reported metric… visibility into the agile transformation was now being provided at the highest level of the company.
2017
As of July 2017, the momentum of the Agile transformation at Fannie Mae spans both business and technology. There are now 220 Agile teams, four proven Agile implementation approaches (SCRUM, Kanban, Scrumban and SAFe), as well as recent re-organization to align to nine customer journeys, all of which is supported by corporate functions focused on adopting lean management values and practices. As part of its continued support of the enterprise transformation, the Agile COE is focused on four key areas, specifically:
- Engaging with the portfolio leadership to provide the tools and support to drive the Agile maturity of their respective portfolio. To do this, the Agile COE provides the portfolio’s VPs and Directors with key metrics to understand their current capabilities and then, working with their Agile Coaches, developing a strategy to improve the portfolio’s overall maturity. This effort is repeated monthly to ensure strategies/tactics are aligned and targeted to achieve the portfolio’s transformation goals.
- Supporting and maturing teams’ Agile practices by providing both standard and customized training, both for new and maturing teams, maintaining resources to promote knowledge-sharing, as well as by providing experienced agile coaches possessing specialization in areas of focus e.g. Agile engineering practices, business agility. Additionally, the Agile COE helps to establish Communities of Practice (CoP) to extend/share knowledge across the organization.
- Engaging with various supporting processes like the Product Development organization, ePMO, HR, Business Architecture, and various governance bodies, to continue improving organizational alignment with Agile processes.
- Communicating Agile/Dev Ops successes to continue to build on the momentum of the transformational milestones achieved to date, while encouraging broader communication within Fannie Mae. Additionally, Fannie Mae is actively involved in the regional, and national, agile community by enthusiastically sharing our experiences and learning from others.
4. Lessons Learned
Fannie Mae has made considerable strides in its Agile transformation. It has proved to be a journey from which we have learned numerous key ingredients that support an Agile transformation – ingredients that other organizations, who are contemplating a similar effort, may find helpful. Our learnings took several forms: circumstantial learning, an outcome that was not intentional; experimental learning, an outcome based on a hypothesis; and validated learning, an outcome that was both expected and intentional. A selection of these learnings are as follows:
Leadership is critical. For Fannie Mae, having senior leadership that understood how Agile works and, more importantly, understood that Agile transformation was essential was pivotal to getting an early foothold from which to launch the agile transformation. Early in the transformation, when integration was difficult, and the value of Agile was questioned by some, our Agile executives were able to change the language from one of failure (the typical response) to one of learning. This change in mindset not only provided room for the teams to mature, but, more importantly, started establishing the legends that were critical in solidifying the benefits of Agile. The constant support, evangelism, and air-cover provided by the Agile executives was key to changing the dialogue within the organization.
Middle management is the challenge/opportunity. Senior leaders quickly understand the benefits of Agile and like what they see. Teams quickly learn how to work together and, for the most part, like the team concepts which support an Agile implementation approach. However, middle management is often left-out and, on their own, struggles with how they fit in. This is problematic because middle management has traditionally been held accountable for the success of the overall program. Furthermore, middle management has, traditionally, achieved their status by being effective at getting things done. Agile, however, advocates for the creation of self-managing teams who are responsible for their own delivery. Suddenly, middle management is left trying to figure out their role, and in the absence of direction, will fall back to the same command and control behaviors that made them successful. Middle managements failure to pivot their behaviors to support the Agile transformation is the quickest way to kill the Agile mindset. When middle management is on-board, demonstrating the values and behaviors of Agilists, the rest becomes easy; otherwise you might not ever achieve the Agile mindset.
Find something complex and visible. At Fannie Mae, the overarching focus has always been on how to mitigate risk, particularly with respect to large technology projects. Our track-record was mixed at best. While people inside of Fannie Mae had tried for years to get Agile adopted, they met with little support. There was always a perception that there were credible reasons why “agile doesn’t work here.” However, the CSP program implementation that established Agile’s credibility within Fannie Mae was a very large, complex, and risky endeavor. Despite this complexity, Agile showcased the risk mitigation benefits of delivering working software iteratively as well as integrating early and often along the way. The visibility created by a high-profile initiative helped to convert executive leadership who then became key allies in the broader transformation.
Focus early on supporting processes. In traditional organizations, everything e.g. governance, budgeting, recruiting, vendor strategy, is centered on a phased development approach. While you can’t address all of these supporting processes quickly, focusing early on these areas can accomplish two critical objectives. First, it signals to your teams that the organization is serious about changing the way of working. Second, it helps to effectively streamline the process itself thereby further helping to build the case for change. One of the first things we focused on changing was our SDLC. The result was a development process that was much more conducive to Agile. Admittedly, the first pass wasn’t perfect – we have already updated it two additional times. But the fact that it changed at all gave a visible message to the organization that we were, in fact, changing. Since then, we have focused on optimizing the governance processes, changing the way we recruit and title employees, changing our funding precedent to a capacity-based budgeting model, redoing facilities to support colocation and collaboration, and updating our reporting and management tools to support an Agile environment.
Don’t be too prescriptive, except where you need to be. This is an area where you really have to be alert to the culture of your organization. At Fannie Mae, we focused on a few core practices e.g. two-week sprints, optimally-sized teams, dedicated team members and good colocation. Beyond these few practices, we allowed teams and portfolios the opportunity to explore and learn. This meant that adopting Agile values, principles and practices was each team’s responsibility, not management’s directive. The teams determined the values, principles and practices they would use to achieve their goals. The organization provided the necessary subject matter expertise to support the teams as well as inform their decisions regarding different tools and techniques.
An Agile Maturity Model (AMM) can really help advance the effort. When we started, we needed to understand how mature the teams were in their adoption of agile principles and practices in order to help guide our coaching efforts. Little did we understand the numerous benefits using the AMM would provide. First and foremost, it helped teams and management understand that Agile is a journey and not something you can simply train and then immediately perform perfectly. Secondly, and equally as important, it gave the organization a common language on how to talk about Agile maturity as well as set strategic and tactical objectives. Lastly, even though we did not define a target state or timeline for advancing in maturity, teams challenged themselves to increase their maturity independently using the AMM as their benchmark.
Figure out how to get the business engaged. At Fannie Mae, our leadership established a key strategy to be more customer-focused and to achieve this through lean management techniques. To help introduce lean management, our business partnered with a mainstream strategy consultancy. This firm introduced Lean Start-up concepts and design thinking approaches to our product development organization. These techniques established the linkage between business objectives and technology development. The fact that it was a business-driven effort, with a business-centric strategy consultant, enabled our business to rapidly change their thinking. Without them, the technology organization’s efforts to engage the business would have been much slower and we would have risked having the transformation stall and, ultimately, fail.
Once you have momentum, move quickly. Change is difficult and tiring. The longer the transformation takes, the more difficult it is to maintain focus. Once you lose management’s focus, the benefits achieved will quickly start to erode.
Keep your technical excellence focus. We started off our journey intent on creating process capability using Scrum. While we focused on developing in the same common mainline, we did not focus as strongly on some of the core Dev Ops practices. In hindsight, the technical debt we were creating took much longer to clean-up than if we had focused on adopting Dev Ops practices from the start.
Dev Ops and test automation is critical. While we did not start here, we did have an organization that was focused on establishing a standard Dev Ops tool suite. As a result, when the teams started rapidly coming into the Agile environment, the Dev Ops tool suite, and associated testing tools, had already been vetted and on-boarding was easier.
Leverage the Agile COE to support, not own, the transformation. The Agile COE should have firm, and impartial, footing in both the business and delivery camps of the organization. Further, the COE should be high enough in the organization to influence and support the transformation as well as move organizational challenges impeding the transformation. However, the organization’s portfolio leadership team should be accountable for owning their respective transformation. Aggregating the findings from the AMM and the AHR, the Agile COE can then work with the portfolio’s leadership team to determine where additional support is required to help the portfolio achieve its strategic objectives.
Find your stories. One of the most important things to do is to find the stories that resonate across the organization. For Fannie Mae, it was the series of legends that resonated with executive management that created the long-standing momentum supporting the transformation.
Lastly, get external support. Transformation is difficult and what seems to make sense in the beginning of an agile transformation can often result in unexpected and costly bad patterns that don’t show up until later. Engaging an experienced Agile consultancy partner, with large-scale transformation expertise that can not only help drive team maturity, but organizational maturity, across different areas of focus e.g. technical agility, business agility, process agility, is critical.
In summary, an Agile transformation at the team level optimizes locally and is fragile. Teams work diligently to mature Agile capabilities and technical practices in pursuit of producing higher quality software more dynamically. Simultaneously, business engages as a partner in the day-to-day development process to define and prioritize features that are of the highest value to stakeholders. At this point, we celebrate our success, send our consultants home, and move on to the next big organizational initiative. A year later, there are islands of capability, but the journey itself has stalled. An Agile transformation at the program level optimizes locally and is also fragile. At this level, progress has been made addressing the technical complexity created by multiple teams working as part of a system and the business has established a strategic vision and product roadmap to guide the ever evolving solution; Agile can now be implemented at scale and the transformation is declared complete. Unfortunately, while some Agile capabilities and practices may persist within the system, eventually, traditional organizational processes and behaviors will overwhelm the program and, inevitably, the journey itself will stall. In order to be sustainable, an Agile transformation requires time, investment and broad commitment to fundamental change of the ecosystem supporting the enterprise, all of which requires the ongoing support of management and senior leadership. If the ecosystem supporting the agile transformation is fragmented and weak, our ability to change the organization will be fleeting. Alternately, if the ecosystem is robust in its ability to support the agile transformation, our efforts will achieve and sustain the promise of Agile.
5. Acknowledgements
This Experience Report is being submitted both as a testament to, and in recognition of, the efforts of all of Fannie Mae’s employees and consultants over the last two-and-one-half years to either directly, or indirectly, support the organization’s Agile transformation to date. Additionally, we are grateful to David Grabel and Rebecca Wirfs-Brock, our Agile2017 designated “Sherpas”, for their on-going patience and advice leading up to the publication and presentation of this Experience Report.
Copyright 2017 is held by FANNIE MAE.
