About this Publication
With this paper, I’ll share the biggest mistakes I made as a Scrum Master while applying Agile principles. I’ll also mention the consequences of those mistakes and how I’ve overcome them! Short iterations by default, not breaking things small enough or lacking of project management are some of the topics covered. In the end, more important than preventing mistakes, is to learn from them!
1. INTRODUCTION
It’s no longer new to anyone that one of the best sources for practical learning is based on the mistakes that we make and how we reflect on them in order to improve our performance in future similar situations.
It was based on this principle that I decided to share my experience. However, this experience is not just about telling you some of the most “valuable” mistakes I made and how I learned with them. From my point of view, what makes this experience even more thrilling is the fact that these mistakes were made when I was trying to find solutions (some of them wrong) to some specific problems. Those “wrong answers” resulted in a new problem, which allowed me and the teams to rethink the adopted solution in order to find the “right answer” to the original problem.
This experience showed me in first hand that the solutions that seem the most obvious ones, not always are the most appropriate ones. Precisely because they seem obvious, they limit our vision. This is true, no matter how much experience we have in some roles. The proof is that I made these mistakes throughout my career, but mostly during the last 6 or 7 years.
2. Background
I’ve been working as an independent consultant for the last five or six years, although I have my own small IT company: DO IT AGILE.
I’ve been participating in missions involving organizations with different sizes and from a wide range of business areas, although the majority of them are part of the European Commission universe.
The missions vary a lot in terms of size, complexity, objective, etc. But I can say that the majority of the mistakes I describe here where made while playing the role of a Scrum Master and/or an Agile Trainer/Coach in each specific project. It’s important to mention that in those particular missions where I acted both a Scrum Master and a generic Agile Coach, I was able to get a better understanding of the teams’ dynamics, which allowed me to be more efficient and effective while performing my tasks.
While I go through the small stories in the next section, I have to say that the biggest challenge (for all of them) was having the ability to, together with each team, recognize that the solution found to solve the initial problem was wrong. This was not because there was any kind of problem admitting that we were wrong. It was simply because in most of the cases, it was not easy at all to find the root cause of the problem, as you’ll have the opportunity to confirm.
3. The Stories
The story that I have to share is actually a set of three different stories. It’s a fact that some of them have as a common denominator a wrong solution for an initial identified problem which, at first glance, looked simple to solve.
Each of the stories represent a different project, in a different moment, with a different team and whose reality ended up developing different types of problems.
In each story, I try to follow the same structure, so that it can be easier to identify and compare key elements, the problems we faced, the solutions adopted and their consequences, lessons learned, etc.
3.1 Story 1 – Too short iterations and “Stepics”.
This next story happened within one of the main European Institutions. The IT department of one of the Directorate General was in charge of development and maintenance of software applications that would support the business. To achieve this, the department had a pool of resources, from which a subset was selected to create a team every time there was the need to develop a new project. This was exactly what happened in our case.
The team had virtually no experience working as such. Some of the team members had already worked together but in very occasional situations. Their experience with agile methodologies was also very limited.
Regarding the project, it was a strategic one, with high visibility. The product owner and Stakeholders commitment was very strong and the pressure to have things right on the first try was also very high.
3.1.1 Story 1 – First Learning: Iterations don’t always have to be short
I used to have a belief regarding the size of an iteration that, no matter what, IT HAD TO BE SHORT. Indeed, the Agile Manifesto, through some of its principles, advocates this. This is clear in its third principle: “Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.”
Now, if you analyse this principle carefully, you’ll see that it also leaves the possibility to longer iterations. And this is not by chance. In fact, there are some factors that need to be taken into account when selecting the size of an iteration:
- Team’s experience using Agile methodologies: The more experienced the teams are, the shorter the iterations can be;
- Stakeholders’ experience using Agile methodologies: The same as for the teams;
- Duration of the Project: The shorter the project, the shorter the iterations must be in order to ensure frequent interaction with the stakeholders;
- The cost of iterating: In theory, the “heavier” is the iterating process, the less you want to do it. In other words, the more complex or formal is the process (artifacts, ceremonies, etc), the less you want to repeat it;
- In this project, at least the first three items were playing a big role, which, as you’ll see, made us take the wrong turn at a certain moment. Now, you may also ask what the right size of an iteration is. Well, you need to make sure that the iteration allows:
- The business expert explain the details of the features to be built in the iteration;
- Developers to code and build the features;
- Collect feedback from testers and stakeholders;
- Developers to incorporate feedback;
- Repeat the last two steps until the stakeholders feel the features are ready to go.
This means that if only iterations with 3 or four weeks allow you to fulfill all these principles, while taking into account the previous factors that I described, then be it.
3.1.1.1 The problem
After the project started, it didn’t take too long for the team to start running behind in every single iteration. It always looked that, at the end, there was too much to do at the same time. We didn’t realize that the lack of experience from the team and from the stakeholders, along with the ceremonies, was taking us some time. But on top of that, iterations were indeed too short (two weeks) and we didn’t realize that either. This made us think about something that we needed to do in order to solve this apparent productivity and organization related issue.
3.1.1.2 Proposed Solution (the mistake)
Because it seemed that we were not being productive enough (we were not being able to perform the several activities to implement a story), we decided to pipeline the work (see Figure 1). This means that, instead of having everyone working on a story until it was finished, each team member was now focused ONLY on the tasks related to his skills (Analysis, Development, testing, etc). This caused that every single person could be working on a different feature in each iteration.
Figure 1. Pipelining work
3.1.1.3 Results
One of the immediate results coming from this approach was the delay in delivering value to the stakeholders, as we were taking several iterations to deliver something completed. But what was even more problematic was the fact that now everyone was not synchronized in terms of features. For instance, imagine a tester going to ask a business analyst questions about a feature that he (the BA) described two iterations ago. Suddenly, the communication between the team members was now seen as an interruption. Because of that and in order to protect themselves from interruptions, team members started to produce documents where they explained how things were supposed to be done! By pipelining our work, we introduced another layer of work which led to even bigger delays. This became quite obvious after the third iteration.
3.1.1.4 Corrective action
After three iterations, we realized that pipelining the work was definitely not the right solution for our problem. During the retrospective, we started looking for answers back to the root cause. As a preparation for that particular retrospective, I raised some points that were related with the team’s activities and outputs taken from the several ceremonies:
- Because they were not always getting all the necessary information and causing some halts during the iteration, I asked them to think about the outcomes from the planning meetings;
- Since they were barely discussing and coordinating work among them, I suggested them to think about how efficiently they were interacting as a team;
- I also suggested them to reflect on how the backlog, sometimes, was not properly groomed by the Product Owner and the stakeholders and what could they do about it.
Based on what was happening, it became clear that the team’s lack of experience with agile (Product Owner included) was actually having an impact in our throughput. When we realized that, we acknowledged it and decided to increase the size of the iteration from two to three weeks, while keeping the same workload per sprint. We would see then if increasing the length of the iteration would help us getting back on track.
After one iteration targeting these topics, the changes were obvious. While still improving, the team was also able to find their own pace and complete all the work assigned to the iteration.
3.1.1.5 Lesson Learned
Although in theory I knew that there are some factors that can impact what you decide in terms of iteration size, I’ve always pushed to small iterations (usually no more than three weeks) since I actually believed that it should be enough. But in this case, reality has shown me something different. We definitely should take all the mentioned factors into account when deciding the iteration length and not just apply a standard rule just because, in theory, it’s better to have smaller iterations.
3.1.2 Story 1 – Second Learning: You need to break things small enough
One of the biggest challenges when using user stories is to make sure they are small enough to be implemented in one iteration. A lot of times, we only discover this when we start implementing it and find additional information about it. But an even bigger problem is when you find additional details on a user story that would probably require a split but you actually don’t do it. You keep implementing it like if it there was no real change in the scope or complexity of that story. This becomes even clearer when implementing systems where the stakeholders and the Product Owner are also discovering themselves on the fly the solution they want.
3.1.2.1 The problem
Just like it happened before, the team started having problems finishing the work foreseen for an iteration. For some reason, the stories where always taking more time than what was initially thought. This was clear since during the implementation of each user story, new additional tasks were constantly popping up, mainly because of new and initially unforeseen scenarios. Sometimes it was just a matter of a couple of hours, while others, new user stories surfaced, taking additional days to complete. This pattern was clearer with bigger user stories (estimated with 8 points or more). Frustration started to take over the team and we definitely needed to do something.
3.1.2.2 Proposed Solution (the mistake)
In a similar situation faced before (described in the previous section), the team managed to successfully solve the problem after increasing the size of the iteration from two to three weeks. Because the team was now facing, apparently, the same problem, the initial thought was to apply the same resolution. In other words, we decided to increase the size of the iteration from three to four weeks.
3.1.2.3 Results
After one iteration with four weeks, things seemed to go back to normal, with the team completing almost every user story.
Optimism took over and, because the team capacity increased due to a bigger size of the iteration, we decided to bring more user stories inside. And because we were getting more confident, we were definitely not focused on trying to split the 8 points user stories (or even a couple of them with 13 points). Because we added more user stories to our iteration, we ended up with more incomplete user stories. In other words, our initial problem got bigger. It became clear that the problem was with the initial size of the user stories and we needed to do something about it.
3.1.2.4 Corrective action
Having realized that increasing again the size of the iteration wouldn’t solve the problem, we finally discovered that in this particular situation, the problem was related with the size of the user stories. The team was having problems analyzing user stories with 8 or more points. Taking that into account, we decided to have user stories split to the point where anything bigger than 5 points would be decomposed. This made each iteration contents more flexible and easier to manage. As soon as the team started to apply this rule, after one iteration the results were obvious and the team was able to consistently complete (or be very close to that) the scope of each iteration.
3.1.2.5 Lesson Learned
It became crystal clear for me and the team what the advantages of working with small user stories are. Apart from giving you more flexibility and control on the work and performance of the team, it will definitely help preventing missing elements in the initial analysis of user stories. After all, it’s completely different to try to depict a bigger user story (8 or 13 points in our case) when compared with a small one (3 or 5 points).
And how do you know if they are too big? Well, from the moment you start seeing user stories needing to be split during the iteration, this is usually a sign that you need to start paying attention to the user stories of that particular size.
After a couple of iterations, we were back to the three weeks size iteration as we were before and we managed to stick to it.
3.2 Story 2 – Project Management and Product Owner
This story happened in the context of one of the biggest telco operators in Europe, where several thousands of people work every day, exclusively dedicated to develop software applications that support the entire business.
The enormous size and diversity of the organization allows the existence of several different approaches to software development, from traditional waterfall up to more agile approaches like Scrum. There were also some initiatives that were already scaling Scrum through the use of the LeSS framework.
The two situations that I describe in this story have their origin in the Product Owner lack of experience and had a direct impact within the team, myself included, obviously. The process transformation that the team was currently going through was the main responsible for this, as they were moving to Scrum as the main software development framework to use.
3.2.1 Story 2 – First Learning: Project Management is not the same as Project Manager
One of the first things you’re taught when you start studying and applying Scrum is that you don’t need project managers. Although this may be true, there’s a problem that a lot of people don’t realise until they step on it. Just because you don’t need a project manager, it doesn’t mean you don’t need project management. And this is where usually problems start to surface. Most of the project management responsibilities are split through the several team members, which means that if something is not properly taken care of, you will run into trouble. In this learning, you’ll see how a simple task that was in “no man’s land” almost terminated the project.
3.2.1.1 The problem
The team was performing well and was delivering value at a constant pace and on time. The Product Owner and most of the stakeholders were happy with the work done so far. Suddenly, the sponsor (a group of two people that were never part of the project initiatives) just stopped funding the project! They mentioned that they were not aware of the results and the progress made so far and because of that, they decided to stop funding it. As you can imagine, this was a shock for the entire team. After all, how was that possible? All the information was properly radiated and was available to be checked in team’s room. So, how wasn’t it getting there? Well, the reason was quite simple: since the sponsors were not visiting the team’s room and since the reporting responsibility was not clearly defined, everyone assumed that it was being taken care by someone. The result: Sponsor without any information for almost three months.
3.2.1.2 Proposed Solution
Since the information was available, it would be just a matter of agreeing on how to properly disseminate it and make sure that the sponsor would receive it. After discussing with the Product Owner and the sponsor, we decided to have a fifteen minutes meeting at the end of each iteration to give the current status of the project, both at the iteration and release level. With this, we ensured that the sponsor had all the necessary information, at the right time, to support any educated decision that he would need to make.
3.2.1.3 Results
Naturally it didn’t take long until the sponsors started to get the information they needed. Actually, it had such an impact that they decided to start attending the review meetings at the end of each iteration. The funding was quickly resumed and the project continued without any additional issues. Although the 15 minutes meeting was kept, it was now more focused on a more strategic level.
This important project management task of disseminating results to the sponsors was no longer an orphan. It was now mine and the Product Owner’s responsibility.
3.2.1.4 Lesson Learned
We can take several lessons from this particular learning. The first one is that even if you decide not to have a project manager, make sure that all the project management tasks are completely assigned. The last thing you want is to have one or more responsibilities ending up in no man’s land.
Additionally, you always need to make sure that ALL the people with a vested interest in the project are kept informed of the progress, especially the sponsors. So, before you start, check if the sponsors can easily have access to the progress information. If not, agree among the team on who and how will the information be radiated to them.
3.2.2 Story 2 – Second Learning: Product Owner CANNOT be a doormat
One of the most important roles in a project using Scrum is the Product Owner. He must be available, since the success of the project depends on how quickly and effectively he can respond to requests. He must be business savvy and have good communication skills as the suitability of the final solution depends how he can explain the needs. But, even more important, the Product Owner must be really empowered by the stakeholders he represents and he must be completely aligned with their goals and expectations. This is what allows the Product Owner to take the necessary decisions and to ensure that they keep the team on the right track, building the right solution.
3.2.2.1 The problem
After an effective good start, suddenly, our PO started to become less and less available to help the team, becoming more difficult to reach. We also realized that he was becoming less aligned with the stakeholders and at the same time, he started to become less and less empowered, almost like a puppet.
As time progressed, we started to have serious problems in moving forward since a lot of clarification and a lot of decisions were needed and the PO was not there to help us. We were running late almost on a daily basis.
At a certain point, we realized that we needed to do something in order to be able to proceed, since getting the PO back looked like an impossible mission.
3.2.2.2 Proposed Solution (the mistake)
When we were faced with the PO’s lack of availability, me and the team decided to start making assumptions when we had doubts. Like this, we wouldn’t have to stop and keep jumping from one user story to the other. It seemed the right thing to do. We knew that this was somehow a risk. But at the same time, we were confident that we already knew enough that would allow us to proceed with a solid degree of confidence. Well, we were terribly wrong.
3.2.2.3 Results
It was during the third review meeting that our decision to proceed without getting inputs from the PO came to get us.
During this meeting, we realized that our assumptions were not correct and that the PO had other ideas. But the worst was still to come. When the PO was explaining his ideas, the stakeholders interrupted him saying that he was not right.
After that meeting, some of the stakeholders lost their trust in the PO. Because of this, they started going around him and making requests directly to the team. It didn’t take much time for the PO to become powerless and with little or no interference! Chaos was installed. With a less than available, misaligned and powerless PO, there was obviously a huge risk for the project. We needed to do something.
3.2.2.4 Corrective action
After a couple of weeks, and with chaos installed, we decided to chase the PO until we could discuss this situation. When we finally managed to get to him, he acknowledged and agreed that the situation became unmanageable and that we should all discuss the situation, including the stakeholders and the sponsor.
It came to our knowledge that the Sponsor gave other tasks to the PO which prevented him to be fully available to the team and the stakeholders, causing some interference in the normal working routines of the team.
After explaining the sponsor how important the role of the PO was, not only for the team but also for the stakeholders, he reviewed his priorities and released the PO from the new tasks, allowing him to get back to the project.
Having this solved allowed the team to continue without further delays.
3.2.2.5 Lesson Learned
One of the first learning that we take away from this experience is that the team should not try to take for themselves the role of the Product Owner (especially the Scrum Master). The reason is quite obvious. Making assumptions without the PO validation can be a deadly combination.
A second learning is that you should always communicate to everyone who has a vested interest on the project the key roles and responsibilities. This will help preventing situations like the one that happened, where the PO was pulled off his duties.
3.3 Story 3 – Too optimal days
This last story that I’m describing took place in the heart of one the European Agencies.
To tackle the need to evolve one of the core systems of the agency (among the several dozen that are managed by them), the board decided to set up a specific team. This team included resources that had never worked together in the past. Some of them have worked previously in other projects developed by the agency, while others, like me, it was the first time.
As a methodology, the agency was using RUP, always promoting a constant iterative approach, delivering value to the stakeholders.
Although the team members have never worked together in the past, they had some experience using Scrum, while others were familiarized with RUP.
Also important to mention is the fact that the project had several rigid time constraints. This was due to the new requirements from a specific piece of legislation which was coming into force on a very specific date.
3.3.1 Story 3 – First Learning: Working days don’t really have eight hours!
One of the most common ways to estimate velocity for your iteration when you start a project with a new team is to make a forecast. This is true, especially if you can’t afford to run a couple of iterations to get some concrete data. No need to say that using historical data is out of the question when you have a new team, for obvious reasons.
When using forecasting, during the planning meeting, you estimate the number of hours available in an iteration for the team. After that, expand each user story, by splitting it into tasks and then estimating each task. This gives an idea of how long each user story will take, allowing the team to have a more clear view of what will “fit” inside the iteration.
One of the most common sources of planning errors comes from the misconception that the number of fully productive working hours per day for a resource is eight hours. This is something that may jeopardize your iteration. In reality, there are already several well-known techniques, like the Pomodoro technique (Cirillo), which supports no more than 6 hours of productive work for an 8 hours day.
3.3.1.1 The problem
During the first planning iterations, and because we didn’t have any historical data nor the possibility to wait for a few iterations to collect some velocity data, we had to make a forecast based on the team’s capacity for each iteration. For this, we took into account the number of public holidays, people that were on holidays, etc. We also considered that each team member had eight hours a day available.
The team depicted each user story for the iteration and split it in tasks, providing then estimates for each based on the optimal days technique. This technique states that the estimates refer to the amount of time taken to complete a task, assuming that you’re 100% dedicated to it, with no interruptions. In a nutshell, we used optimal days as a way to estimate tasks and at the same time, considered each resource to be fully engaged eight hours a day while executing those tasks. And this is where the problems started.
Right after the first iteration started, and in a very mysterious way, the team started to run behind, with a constant pattern, on a daily basis. Progress was always under the initial estimates and the delay was growing at a steady pace. Definitely something was wrong. But our biggest challenge was to understand if this was a problem either with the estimates, either with the team availability, etc.
3.3.1.2 Proposed Solution
It was during the second iteration that I decided to start doing some investigation on this topic. I already knew several people that were applying several principles and rules that prevented them from assuming 8 hours a day as fully productive days. Nevertheless, I needed a little bit more evidence that indeed this was the way to do it. And this was the moment where I came across the Pomodoro Technique. This technique supports an approach that encourages small fragments of work, followed by a mini break. Practically speaking, this would mean, indeed, that within an eight hours day, something around six hours would be the real outcome of it.
After deciding about this, I discussed the technique with the team and I was surprised that some of them, even without knowing it, were already applying it, since it was their way of working. So, without any further delay, we decided that for the next iteration planning, we would consider 6 hours of available work force per day, per resource, to determine team’s capacity.
3.3.1.3 Results
After updating the team’s availability to consider the 6 hours a day, the result was that less work was being included within each iteration, which was now much closer to reality.
Please note that this adjustment didn’t make the team more productive from one moment to the other. What happened is that the team was now being able to complete the work that they have committed in the beginning of the iteration, with an obvious positive impact in their morale and the trust from the stakeholders.
The release plan was now a more realistic plan, with more realistic expectations.
3.3.1.4 Lesson Learned
In this particular situation, I believe it makes sense to talk about what the practical experience just confirmed. The team didn’t become really more productive. What happened is that we assumed reality in terms of workload and reflected it while determining team’s capacity. For me it became now clear that one that works eight hours a day, will actually contribute with 6 hours of effective and optimal work.
4. What We Learned
During this report I presented several lessons learned that were collected during the execution of each project. Although each one of them describes a different learning (based on the specifics of each situation), the global outcome is that, no matter how much of an expert we are, mistakes will always happen. This means that you’ll need to be prepared to:
- Acknowledge those mistakes and share them with the team;
- Search, together with the team, the best solution for each mistake;
- Learn from the mistakes so that you and the team don’t make them again in the future.
As a final observation, I would also recommend that every time you get involved with a new team, you always share the mistakes you made in the past and under which conditions. This will allow the team not only to be aware of those mistakes (and able to identify them) but also to understand their root causes, which, in my opinion, is the best way of learning.
5. Acknowledgements
I would like to start by thanking my family for all the support they always gave me without asking nothing in return. Preparing this conference was just another example of their help and commitment.
I also would like to thank to Susan Burk who helped me improve my proposal for this report during several weeks and iterations. Without her, I wouldn’t have had the chance to start writing this Report.
Last, but certainly not least, Chris Edwards, for his precious help in making this a great and appealing report! Without his contribution you wouldn’t probably be reading this!
A big thanks to all of you!
REFERENCES
Cirillo, Francesco, Corporate Cirillo website, https://cirillocompany.de/
Copyright 2017 is held by Tiago Palhoto.
About the Author
Experience Report
Author:
Aug 2017, Agile2017 Conference
