Many organizations have chosen SAFe as the vehicle to drive their enterprise wide transformation, and whilst SAFe is a broad opinionated framework that gives answers for much of what you will run into, there is also a lot it does not say or does not have an opinion on. This paper gives a glimpse into what we have learned from being 2 years into an organizational change using SAFe, where SAFe helped and where it hindered, what we have adapted and what we have kept, and what we have felt it necessary to add to help us keep moving towards our goal. We will share with you the joy & pain of doing this, of what worked and what didn’t, and what we are still trying.

1.     Introduction

In the fall of 2015 the CTO of SimCorp took a leap of faith and decided that the development organization of 500+ people would ‘go agile’ and that SAFe was the chosen vehicle to facilitate the change.

Whilst there are obviously many differing opinions around SAFe, for traditional companies that have grown used to inventing their own ways of doing things the prescriptive nature of a framework like SAFe can actually be an advantage when you don’t know where to begin.

However, each company is different and the context in which they operate is different, and so there is no one size fits all solution. Regardless of what your starting point is or which framework you choose to adopt, there will always be a need to adapt parts of it to fit your needs and so what follows is some of our story of how we tweaked and adjusted SAFe, and what we added and what we took away to make it better fit our needs and to achieve outcomes.

No matter if you are just starting out with SAFe, or have been using it successfully (or even unsuccessfully) for some time, we hope that at least some of what we share in this report will be helpful to you.

 

2.     Background

SimCorp are the leading provider of integrated investment management solutions and we are proud that almost 50% of the top 100 global investment managers use our product. We are a global company of approximately 1500 people, of which around 500 are in the development organization split across Copenhagen, Kiev, London and Bad Homburg in Germany.

It would be fair to say that until recently SimCorp would have been considered a traditional top-down, command and control type organization who followed a strict waterfall development process, but despite this we were a successful company with happy customers, and who delivered on our promises year after year. Not everything was broken but there was a feeling amongst some that things could be so much better.

It was also decided that whilst everybody had job security, this was not an attempt to downsize, not everybody had role security. Many of our previous roles would be replaced by new agile roles such as Scrum Master, Product Owner and Release Train Engineer which people had to apply for, but as most had no agile experience this meant many had to learn on the job.

2 years later we have 55 cross-functional teams in 7 Agile Release Trains (ARTs) across 4 different countries, all of which have radically changed how they work. What also makes our journey interesting, and in some ways unique, is that we have all 500 people focused on delivering a single product. Of course we are still a long way from perfect, and we are striving to improve on a daily basis, but we definitely have reason to be proud of what we have achieved so far as the results we present at the end show.

3.     Our Story

What follows is the story of our experience so far, told by 3 Enterprise Agile Coaches, one born & bred in SimCorp, the others with years of knowledge and experience from outside and prior experience of working with SAFe & LeSS. It documents what we have learned from using SAFe as a vehicle for large scale change, what worked, what didn’t, what we have added, and what we have changed. Of course, there are many other changes that have been made along the way, but those included in this report are those we consider the most important.

The report is organized such that each change or initiative is in its own section under its own heading, each of which is largely independent from the others, and is in no particular order of importance. So should you wish to try some parts but not others, we can’t think of any good reason to advise against it. By sharing this we hope we can make your journey to excellence smoother, but it also gives us a chance to learn, so if you try something and find ways to improve it, we’d love to hear from you……here goes.

3.1       Big Room Planning for real

Big room planning, or Product Increment (PI) planning as it’s known in SAFe, is seen by many as the heartbeat of the organization. This event is recommended to span 2 days and is the stake in the ground around which all other activities revolve. It’s also a great opportunity to regularly get everybody and have some fun!

For most organizations adopting SAFe the first PI planning will be the launchpad into their new world, and so we think it’s vital to make a good first impression. SAFe provides a very detailed hour-by-hour plan for how to run the event which largely makes sense to get going with, but there are some of the recommended sessions that we have seen fail each time we have launched a new ART and so we hope the simple advice that follows can help getting you off on the right footing. Here is our story of running big room planning for real.

There are some sure-fire ways to drain the energy from a room of 100+ people, and having one person address the room from the front for long periods is one of those. Unfortunately this is the format SAFe advises for many of the sessions that it recommends. The sessions in question are Business Context, Product Management Vision, Architectural Vision, and Draft and Final Plan Reviews, and whilst we do not question the validity of the intention of these sessions, it’s the recommended format that we caution against. Fortunately it’s easily fixed so here are some reasons why we found things didn’t work, and how we fixed them.

Kicking off a 2 day event with 3.5 hours of presentations on Business Context, Product Management Vision and Architectural Vision as SAFe recommends is a tough sell, no matter how interesting or entertaining the presentations are. Having seen this approach fail each time we tried it we have now found better alternatives. We still believe there is value in kicking off PI planning with an inspiration or visionary session to set the context but it should be kept to a minimum. People are keen to get going with the real reason they are together which is to collaborate and plan. So how to ensure the vision and context is clear? In most cases we now just run them as online sessions in the lead up to the PI planning, and we also record them so those that cannot attend can consume them at their leisure. Not only does this cut down the amount of one-way presentations at the PI planning, but it also gives people a chance to digest what they have heard and ask for questions and clarification.

For the Draft and Final Plan Reviews, we found that having a representative from each team stand in front of the room and recount their plans, especially when there are sometimes 14 or 15 teams, not only took a very long time and often contained too much detail causing people to lose interest in proceedings, but more importantly it stifled the valuable discussions that arise when a new set of eyes questions why certain decisions have been made or why a particular approach to a problem has been chosen. We observed that many people would not feel comfortable raising questions or challenges in front of the whole room, and even when they did it was very hard to facilitate a good discussion around it resulting in valuable input being lost. We have experimented with two different approaches that we feel work much better and are used in the organization today.

The first thing we tried was the idea of a Market Place. Using this format we would for example have 5 sessions of 10 minutes during which the teams or individuals can choose which teams they visit to review their plans. This requires that at least one person from each team stays behind to explain their plans, and it obviously means that not everybody gets to see all plans, but the benefits we have seen are that the discussions happen in much smaller groups which leads to better quality feedback, and people get to visit the teams that are working on things that interest them so they are generally more engaged.

The second approach we have tried with good results is the idea of Sister Teams. In this approach teams pair up and commit to doing a thorough review of the others plans, receiving the same in return. The benefit of this over the Market Place approach is that it’s guaranteed that at least one other team will be rigorous in challenging your plan, whereas with the Market Place it’s more based on the interest of others so there is a danger that the topics people find less interesting do not get challenged sufficiently. Allowing teams to decide who they should pair with also increases the chance that they have an interest in the work of the team they will review, giving a better chance of leading to a valuable outcome.

The last of the sessions recommended by SAFe that we caution against using the recommended format is the Confidence Vote. Whilst not a presentation per se, it is still a session run in plenum across a large number of people. We found that whilst most felt able to cast an informed vote for their own teams’ plans, or the teams they were closely associated with, they found it hard to do the same for the plans of the whole ART. We also hypothesize that when voting in front of the whole room, they are less likely to give a vote (e.g. 2 out of 5) that would require them to explain their reasoning to everybody. To solve this we have now abandoned the ART level confidence vote altogether and now only conduct confidence votes at the team level, finding this much more beneficial and with evidence that low confidence votes and re-planning do actually occur.

We hope by sharing these simple insights that with a few simple tweaks it will ensure your first, or future PI planning sessions run much more smoothly.

3.2       Feature Backlog Refinement

Of all the challenges we have faced, Feature Backlog Refinement is the probably the one that we struggle with most consistently across the organization, and is one that we are still working hard to get to a place we are happy with. Our experience, and what is also likely to be the case for many teams or organizations transitioning from traditional ways of developing software to becoming more agile, is that one of the first things to be dispensed of are the upfront designs and specifications. Previously we had people who would in some cases spend many months painstakingly writing requirements specifications and design documents, getting them reviewed and editing and tweaking them. Then then along comes the agile manifesto with its ‘working software over comprehensive documentation’ and suddenly we think we can do away with any upfront thinking and do everything ‘now’.

For us this manifests itself as anywhere up to 100 people or more arriving at a 2 day planning event and being presented with a wall of loosely defined features, some being just one liners that had surfaced a few days before the event, which few people outside of Product Management had previously seen. The result of this is that rather than teams spending the two days of the planning event engaged in good discussion, slicing features into stories and making plans they believed in, they instead spent the time trying to understand exactly what was expected of them and many of the answers they needed not being forthcoming. Often teams would have circular discussions around ‘this item could be really simple, or it could be really hard’, which meant that they spent their time creating a 12 week plan which they had no faith in, where the first sprint contained a number of spikes to drive out uncertainty, which eventually resulted in a wholescale re-planning effort after the first sprint once the problem and the technical constraints were truly understood.

Therefore our experience is that the shift from trying to define and plan everything upfront, to defining and planning almost nothing upfront is too extreme and a middle ground needs to be found.

We believe that the solution to this problem is to institute a Feature Backlog Refinement process. In Scrum its generally accepted that 5-10% of each sprint should be used to refine the backlog for upcoming sprints, and we believe a similar approach is needed in SAFe, both at the program level for features, and at the portfolio level for epics (although this is still something in the future for us). After all, it’s planning three months of work for over 100 people, which is no small feat.

So what does this mean in practice? Well, SAFe has little guidance to offer, except that ‘features should be refined and socialized with the teams’. The Feature Backlog Refinement process that we’re fine tuning, starts in the week following the PI planning event. The Product Manager, Product Owners (in the PM CoP) & System Architect present a prioritized backlog of Features & Technical Enablers for the next PI. At this stage we acknowledge that some will just be a headline and not much else, and that inevitably things will be removed and others added over the coming weeks as we learn more and business priorities change, but we hope for around 50% certainty in the initial backlog. We are already seeing some pockets of success with this with the key being that the Product Manager has taken ownership of the process and has scheduled frequent recurring meetings to prune the feature backlog. This enables them to always keep the backlog for upcoming PIs to a realistic level, rather than piling it full of features that need to be sorted through just prior to the PI.

Once the Feature Backlog definition is under control, the next phase of the process to institute is the refinement of the Features, with the goal of 20% of Features being refined in each of the 5 sprints to ensure continuous refinement and to smooth the flow of work. To facilitate this we are experimenting with introducing the concept of a ‘Virtual Team’ in each ART which consists of a representative from each team in the ART, the RTE as facilitator, the SA and possible stakeholders, e.g. customers. The team representatives will distribute the Features among them and take them back to their team for refinement, to the extent they fulfill the Feature Definition of Ready (DoR). Once complete, the representative will report back to the Virtual Team, and the Feature will go into the planning backlog for the upcoming PI. Of course in the process of refinement it will likely be that the team need to communicate with other teams if it’s in an area they are unfamiliar with, but this is a good way to socialize features and for people to become more t-shaped. One of the challenges with this approach is that the team that refines a feature is often compelled to then go on to pull that feature at the PI planning as they are familiar with it and it is well understood and this is something we are still working with, but on balance the benefit of having well refined features far outweighs the downside of having teams have an affinity for certain features.

In closing out this section we’d also like to share an experiment we have tried to help us ensure ahead of the PI planning that there is at least one team willing and able to pull each feature on the backlog. We did this in response to observing on a number of occasions that some features are not pulled as no team is comfortable with them. To make this visible and to give us chance to take action prior to the PI planning event, we simply distribute a list of all features a couple of days ahead of the event and ask each team to quickly score each one based on the following criteria:

1. We’d be energized if we get this Feature
2. Bring it on!
3. OK, we can do it
4. No thanks

The results are reviewed within the leadership team and if any features do not have at least one team that scores 1-3 they have time to investigate and take action to fix it.

If you also see similar problems with readiness of features then we hope you find this useful, and whilst we don’t yet have this working successfully in all areas of the organization, we have seen enough to be sure that it is the right path for us to be following right now.

3.3       Impediment Removal Process

Many organizations are not good at fixing their problems. This could be for any number of reasons but common ones are because the problems are not visible, because sign off or agreement is needed from management to change things, because things have always been that way, or as was often the case for us, that it has always been somebody else’s job. Part of the reason for this was that we had over time built up specialist teams who many others relied upon, and so it was always easy to throw problems over the wall for them to fix rather than investing the time yourself to resolve it.

To help us begin to tackle this problem we have implemented an Impediment Removal Process (IRP). The IRP is our attempt at making our organizational impediments transparent to all including our senior management team, to give us a way to ensure we are focusing on the highest priority items, to help us to focus on the need for constant improvement, and to also help decide what not to fix.

The concept is simple. When a team identifies an impediment, they raise it as a Team level impediment and the Scrum Master has 48 hours to try to find a way to resolve it. If it cannot be resolved within 48 hours then it becomes the responsibility of the ART Leadership Team who have an additional 48 hours to resolve it. Failing that it then becomes the responsibility of the Senior Management Team. If they fail to resolve it then it is ROAMed (Resolved, Owned, Accepted or a Mitigated). All of this is implemented in a simple digital Kanban board that is visible for all to see. The IRP ensures that a team will get senior management attention to their problems within a week – and it gives senior management visibility into all problems the organization faces.

Whilst the IRP is a tool for teams escalate impediments they feel are outside their own sphere of influence, we also see it as a tool to allow impediments be pushed back down and for teams to be empowered to fix them. If you have a culture where people are not used to fixing their own problems it can take time to make this adjustment, especially in the early days, but it is possible to do so by encouraging teams to take on the challenge when its possible for them to resolve it themselves.

However, our experience has been that doing this is easier said than done. Whilst rolling out the concept is relatively easy, getting it to stick is much harder. Among the problems we’ve seen are a feeling of not wanting, or not needing to share impediments, a loss of faith in the process when impediments aren’t fixed immediately, especially when and some impediment just be very hard to fix; for example if like you haven’t had a culture of automated testing, then it is very costly and time consuming to resolve the impediment and get that in place.

Despite this and that the process needs constant marshaling to keep things moving, we have seen some positive results with people taking ownership for ensuring impediments are resolved, improvement in the handling of impediments that span ARTs, and the ability to see where similar impediments impact many teams. The IRP has been a key part of raising the importance of the need to remove of impediments in our organization, but it’s safe to say that it is still not part of our DNA and is something we need to persist with until it realizes the potential we think it has.

3.4       The role of the Development Manager

There are many examples of companies who have adopted agile ways of working and have decided to dispense with managers, but in our experience even in agile organizations we need managers. Somebody needs to take care of individual and team development. Whilst the ARTs focus on operations in the short term and what is being delivered in this PI and the next, we need somebody who looks at the mid-to-long term from both a strategic and tactical viewpoint; what capabilities and skills do we need to develop, and what capacity do we need to deliver when?

SAFe describes what a “Lean Agile Leader” looks like, and what the responsibilities of a “Development Manager” include, and it’s an impressive smorgasbord; any organization would love to hire individuals who can cover all of that within one person.

In our reality, it looks a bit differently. The average Development Manager covers 20-30 people or 3-4 teams. Since we cover 4 locations, teams may be spread across locations, so there is a need to decide if the guiding principle should be allocated Development Managers by location or team. Both have pros and cons. So some Development Managers cover team members in several locations, and some teams have more than one Development Manager.

Furthermore, if a Development Manager covers a whole team, it means covering a lot of different skillsets; for example, it could be several different programming languages, test, scrum mastery, product ownership. No person can be an expert in all these areas, so being responsible for the professional development of all team members is challenging.

We’re trying to overcome this challenge by appointing “Learning Owners” for specific areas. For example we have Learning Owners responsible for the different programming languages we support including C#, as well as Architecture, Product Ownership, Agile, etc. The Learning Owner for each area has the responsibility for the learning curricula and training budget in that area, and act as sparring partners for the Development Managers within their specialist knowledge area.

By putting this setup in place we hope to be able to strengthen the professionalism of each role, and put our people onto a path of lifelong learning and support their needs as we evolve into new areas of business and expertise.

3.5       Leadership Teams

In SAFe the ART Leadership Team is a troika; the RTE, the PM and the SE, each responsible for an area, and jointly responsible for the well-being and performance of the ART. They execute, have a focus on this PI and the next. The Development Managers are not per definition part of the Leadership Team, but according to SAFe  “the manager has personal responsibility for the coaching and career development of direct reports, takes responsibility for eliminating impediments, and actively evolves the systems in which all knowledge workers operate. They have final accountability for effective value delivery as well.”

In addition to this are also responsible for building the teams, so why aren’t they a natural part of the ART Leadership Team. Well in SimCorp they are. For us it seemed like the right thing to do, but it does come with some downsides.

We find that ART Leadership Teams are naturally more execution and short term focused, whereas the Development Managers should ideally also be more strategic and have a longer term focus, but by having them part of the ART Leadership Team we find that the strategic and longer term focus is lost somewhat.

3.6       Human Resources (HR)

SAFe has very little to say on HR, but we realized early on in our transformation that some of our HR policies & guidelines that were focused on individual excellence were not aligned with our new desire for teamwork & collaboration, and a focus on value rather than output. Two things that stood out were individual performance appraisals (and the linked bonuses), and detailed time recording. Whilst these were not at the top of our must fix list and were outside of our immediate sphere of influence as they resided in a department with different reporting lines, we are really happy to share the progress we’ve been able to make.

First to go was detailed time recording. Prior to this people had been required to account for every minute of their day, with some people recording time against 10 or more individual items in a single day. Although this gave us a mass of data, we didn’t really use it for anything valuable. The decision to dispense with time recording was straightforward. In one of the regular meetings with our CTO, who was also the sponsor of the transformation, the subject of how to handle time recording came up. Amongst the discussions on how to improve it was a suggestion of ‘why not just get rid of it’. Coming from where we were this was a big step. We liked the illusion of control that detailed time reporting gave us, and so this was a challenge that went right to the core of how we did things, but he got it. After pondering for a few moments, it was possible to see the lightbulb go on. ‘We can do that?’ he asked, ‘sure we can’ was the response. He turned to somebody in the room and said, ‘please look into this and get back to me before Christmas’. That started the ball rolling and now after two long iterations on the process we are at a point where we are happy.

In the first iteration we managed to remove the need for detailed time reporting for most people with a few exceptions, for example we discovered there are legal requirements in some countries that things like holidays & sickness are individually recorded and this is something that we are still obliged to do.

The next problem we faced and one that we have only just agreed a workable solution for, was that we had some agreements with clients that were based on time & materials, and some that were fixed price and so there was still an interest in knowing how much time and effort has been invested in it to determine if it has been profitable or not. For the last year anybody working on anything in this category was still required to log the hours spent, but by using data that we have collected in the last year or so in our online tooling we have now also been able to remove the need to for this. By knowing the average of how many stories we deliver per PI across the organization, and knowing what our operating costs are, we can simply work out the average cost of a story and use this figure in our calculations for billing. By sampling some data we have determined that this approach is no less accurate than asking people to manually register their hours, meaning that other than holidays & sickness there is now no need for anybody to register time. Whilst this may not seem significant in the grand scheme of things, it showed that everything is up for discussion and change and that the amount of effort you put in is no longer the thing we value most as we no longer track this.

Whilst we were in the process of dispensing with time reporting, we also began to tackle is annual individual performance reviews. This was a core part of our company DNA and was also linked to individual bonus payments and so changing it was never going to be easy, but we knew that if we didn’t act then we would be in a situation where individual goals and team goals would not be aligned which could lead to sub-optimizing team performance to achieve individual goals; it is after all well known that you get what you measure.

Although we were in the middle of the annual performance appraisal cycle and everybody had been through their goal setting for the year and had their bonus incentives assigned, with the support of our CTO a decision was made that the follow up on goals was now optional, and all individual bonus payments would be honored in full until a suitable alternative was in place. This was a bold step, but made it crystal clear to all that we were serious about change. In the year since alternative ways of setting direction and defining and tracking objectives have been discussed, and we have now in the early stages of experimenting with Objectives & Key Results (OKRs).

No longer having to record time has been widely welcomed across the organization, and looking back it now seems strange that we did it so diligently for so long as there are no noticeable downsides form doing it, and other than asking people to stop recording time we’ve had to do very little to get this change implemented. Similarly people seem to appreciate no longer having the annual top-down goal setting that we had in the past, and the early indications are that OKRs have been very well received, primarily it focuses on the team rather than the individual, and the teams are in control of setting their own objectives.

4.     Results

Whilst it’s difficult to directly attribute any of our results to the tweaks and changes we have made to SAFe, and whilst we know we still have a long way to go, the results so far have been impressive. When we look at the hard numbers we can see that:

 

• We have reduced the period from finishing development to release from 18 to 5 weeks. Previously much of this time was consumed by an explicit release test period.
• At the same time we have halved our release cycle from 6 months to 3 months.
• Reduced our Feature cycle time by almost 30%.
• Increased the number of stories we deliver by over 30%.
• All the while reducing our pile of defects from being in the 1000s to a much more manageable number in the low 100s (we still have work to do here).
• Including a historically low numbers of customer reported defects.
• And we have achieved this whilst keeping a high level of employee satisfaction (average Happiness Index of almost 4 out of 5).

 

In addition to this we have also observed many improvements that are harder to measure, but which we see as just as valuable. In particular:

 

• People now collaborate much more, both within their teams and across the organization.
• People are open to, and enjoying new ways of learning. We have a couple of active book clubs, Hackathons are becoming the norm, and we ran our first successful Open Space for Scrum Masters in 2017, with more planned for 2018 for both Scrum Masters and other roles.
• People now share much more, successes, things they have tried, good articles.
• People are now much more open to speaking up about problems they see within the organization, and there is a general desire to improve things.
• People now appreciate the benefits of effective facilitation, and we are beginning to open up to feedback form others.
• It has also been a good employer branding opportunity for the company, as we have hosted a number of events for the agile community and had people speak about our journey at other industry events and conferences.

5.     What We Learned

Of course, we have learned a lot along the way and so we’d like to share some of that with you. One of our key learnings has been that no matter how good an idea something is, no matter how much potential an idea has to improve things, unless you have people on board with the idea and are willing to invest the time and energy required to get it up and running and make it self-sustaining, it is very hard to make it a success. This is especially true when trying to scale the idea across the whole organization and so be sure that if you really want something to stick that you have the free time and energy to see it through as a good idea that is never made operational, will just remain a good idea.

Another realization is that people have a limited capacity for change. When you see so much that needs fixing, and so much potential for improvement, it’s sometimes easy to forget that every time something changes it has an impact on people.  These are people who are already struggling to absorb the last round of change; people who are already busy with their day-to-day work; people who knew yesterday how to do their job, who today are now unsure. All of this fights for their attention, and so it is worth paying attention to the amount of Change in Progress at any one time. One way to help reduce the impact on individuals is to run a number of different change initiatives in different corners of the organization. This also gives you the chance to tweak and adjust them before exposing them a wider set of people, and to have good stories to tell and ambassadors who will sell the idea for you. It’s often tempting to roll changes out big bang, when we know it’s often better to start small and iterate. Maybe sometimes we need to pay more attention and practice what we preach.

For recurring events like Big Room Planning, the improvements suggested above have without doubt improved the event, and we would advise that you avoid making the same mistakes if running your Big Room Planning for the first time as it leaves it with a sense of it could have been so much better. However we have also learned that doing exactly the same thing time after time leads to fatigue and the feeling of process theatre, so there is a constant need to keep it fresh to keep people engaged. Much like you would try different formats for your retrospective, keep trying new things for your Big Room Planning and listen to the feedback.

Finally, from writing this paper we have learned that it’s also important to take time to pause and reflect. It’s been a healthy process to step back and put our thoughts into words, and to realize how much we have achieved, and it has also helped us spot a few areas where we could have done better.

Everything we have presented above is from our experience, based on our context, and so whilst we are happy to share this with you the community, in the hope that our experience may smooth your journey, you should remember that just because something did or didn’t work for us, that may not be the same for you. As any great agile coach will tell you ‘it depends’. 😊

So feel free to try it, tweak it, and if you find something better, then please remember to pay it forward.

6.     Acknowledgements

As always there are people to thank for their help, support and contributions along the way. Firstly we’d like to thank our manager Malene Maegaard Krohn for encouraging us to share our story, to our team mate Marianne Garre Fink for her feedback in helping us shape this report and for being there to give our team balance, to those within the agile community who inspire us, there are too many to mention, and to each other for being an awesome team! Finally, we’d like to say a big big thank you to our shepherd Maria Paasivaara for her guidance and encouraging words as we crafted this report.