In this experience report, we will talk about why we built a virtual Scrum Master and share how it has helped Agile teams follow Agile practices. In addition, we believe that artificial intelligence(AI) in Agile will not replace the collaboration between and within the teams but augment it to the next level.
1. INTRODUCTION
Agile is becoming a standard delivery method adopted by organizations across the globe, according to VersionOne’s 11th Annual State of Agile Report. While 94 percent of survey respondents said their organizations practiced Agile, 80 percent said their organization was at or below a “still maturing” level. There are multiple reasons on why the Agile maturity of the teams are low, but the key one is teams look at Agile as a process change rather than a cultural change. At Accenture we have been delivering solutions using Agile practices and principles and based on our experience we do face the challenges within the Agile teams such as the whole team’s limited experience with Agile, slowing down of work due to limited access to Product Owner, incomplete\less detailed user stories leading to high onshore dependency and struggling to keep momentum with continuous churn of Agile events through active participation and to maintain quality of artefacts (backlog, burndown, impediment list, retrospective action log etc.). These challenges manifold when delivery teams are practicing distributed Agile at scale. Apart from these challenges, at Accenture we are focused on how to make our teams more productive and ease the ways of working for the Agile teams.
2. Background
In 2015, Accenture looked to use AI to augment the work of application delivery teams. In fact, we launched Accenture’s intelligent automation platform in April 2016, called Accenture myWizard®. With the launch of myWizard, it introduced a virtual Scrum Master agent. Virtual Scrum Master monitors numerous aspects of Agile development process—consisting of requirements, releases, metrics and resources—alerting the team of any potential issues and providing possible solutions. The virtual Scrum Master also helps Agile teams with the scrum ceremonies such as sprint planning, sprint retrospective and daily stand-ups by providing descriptive and predictive insights.
In this report we discuss our experiences in deploying the virtual Scrum Master and the team feedback where the virtual Scrum Master was deployed. Did the AI infused virtual Scrum Master help to address or mitigate the challenges it was introduced to address? We will also discuss our perspective of the virtual Scrum Master replacing the existing human Scrum Master soon.
Before we jump in, let’s introduce the authors and their role in building and deploying the virtual Scrum Master.
Jeffson Dsouza or Jeff is an Associate Director and is the Agile Community of Practice lead at Accenture. His role at Accenture involves providing Agile support in terms of coaching and mentoring various Agile teams within the organization. He is the product owner of the virtual Scrum Master agent and is responsible for the success and the failure of deploying the virtual Scrum Master across Agile teams.
Raghavendra Meharwade or Raghu is an active member of the Accenture Agile Community of Practice since 2011 and has worked as Scrum Master, Agile SME and Agile Coach for projects spread across geographies and domains. In his current role he is responsible for leading a functional architecting team that sets up the virtual Scrum Master and coaches teams on its use.
3. Our Story
3.1 Why did we build the virtual Scrum Master?
While AI has the capabilities to automate scenarios within the software development lifecycle, the question was which use cases should we look to automate. We talked to Scrum Masters and Agile team members to understand their ways of working through time and motion study process. The study used the below mentioned tabular form to record the automation opportunities within the Agile teams:
Figure 1. Time and Motion Study Template
We sought to understand the time spent on all Agile ceremonies such as product backlog refinement, sprint planning, sprint retrospective, daily stand-up, release planning, definition of ready and definition of done activities and looked-for automation opportunities. Not surprisingly, we discovered numerous automation opportunities for deploying within Agile.
We concluded that roughly 30 – 40 percent of the tasks within the Agile ceremonies could be potentially automated, increasing the efficiency of the team. This analysis was the impetus needed to build and implement the virtual Scrum Master.
3.2 Introducing the Virtual Scrum Master
The purpose of the virtual Scrum Master is to be a catalyst for the whole Agile team and improve their collaboration, speed of delivery and alertness within the Agile lifecycle. Based on the above analysis on the opportunities for automation, we created a list of the following features or key activities for the virtual Scrum Master:
- Monitor the overall project status at the team, sprint, release and at the Agile Release Train (ART) level. Use historical information to predict the future and take corrective measures when needed through predictive and descriptive analytics.
- Measure and auto compute metrics and reporting.
- Act as a scrum guide for the team and assists them during release and sprint planning, daily standups, retrospectives and reporting needs.
- Support requirement management activities.
- Be available for team’s support 24/7
- Perform self-healing routines like change in sprint scope, handling must-have stories.
- Drive standardization of Agile practices while scaling in a distributed fashion by providing alerts and recommendations on Agile leading practices.
3.3 Virtual Scrum Master Functional Diagram
This agent leverages AI technologies to collaborate with team members, product owners, Scrum Masters and even with stakeholders irrespective of collocated or distributed location set up.
Figure 2. Virtual Scrum Master Functional Diagram
As an example, sourcing Agile Lifecycle Management (ALM) tool data, the virtual Scrum Master has the capability to build co-relation between user story quality with various engineering and testing activities that the team is performing. It can monitor Jenkins logs over time and warn team members if it sees decreasing or increasing trends which could potentially harm the team’s outcome.
3.4 Building guided assistance for sprint planning and retrospective sessions
In the Agile ways of working, sprint planning and retrospective are one of the key checkpoints which determine how the whole team is working together and areas of improvement. It is important to make sure that we augment the team with the right amount of support to perform their duties in the most efficient manner during the sprint planning and retrospective.
3.4.1 Assessment of Sprint Planning Process & Outcome
As part of value stream mapping process, we interviewed eight Scrum Masters and asked them capture typical tasks done by their team prior and during sprint planning session, noting objectives behind those tasks and approximate effort spent by the team. We also collected information around challenges in sprint planning. One common challenge was too much focus on capturing each possible task in the sprint backlog. Teams were spending a major portion of their sprint planning session on these activities. While the tasks and the estimates are necessary for the team to understand what it’s committing to, they are tools a team should use to decide on the right set of product backlog items to select for a sprint as the primary goal of sprint planning.
Another major challenge was failure to set aside enough time to spend on unpredictable defect flow. Being unpredictable, teams were struggling to predict defects when compared to prediction of upcoming user stories for a given sprint. Scrum Masters were not mature enough to adjust timing to include upcoming holiday seasons, release cycles or first few sprints of the release vs. last sprint preceded by deployment. In some cases, we found that even the most experienced Agile team found it difficult to forecast defect and predetermine how they will affect the time allocation during the sprint.
Scrum Masters expressed how they are spending a fair amount of time to help teams identify uncertainties associated with part of the sprint backlog which will demand for ‘spike’ planning in advance. One team always had the challenge in terms of assessing depth of uncertainty which will call for the ‘spike’.
A few Scrum Masters noted the difference between planned efforts estimate and actual efforts that teams took to deliver a task. It ranged from +/- 15% to +/-30%. This meant the team either overestimated the required effort to commit the sprint backlog – thus, missing an opportunity to scope more stories. Or the team underestimated the amount of time required which made the team move out stories at the end of the sprint, leading to an unhappy product owner.
We also observed a few teams during sprint planning followed the practice of setting aside a certain portion of ideal capacity towards ‘backlog grooming’ sessions. This cut short ‘available capacity’ further. During these grooming sessions, a few team members got involved in assigning estimates to the new user stories, correcting existing estimates, modifying stories and giving suggestions on how large user stories can be further decomposed.
3.4.2 Virtual Scrum Master – Sprint Planning Assistance
To help Scrum Masters overcome the above challenges, we strengthened our virtual Scrum Master capability to help teams conduct sprint planning meetings and predict tasks and efforts for a given set of stories leveraging Machine Learning capabilities based on historical performance achieved by the team. During subsequent releases, we facilitated bulk task creation in the Agile Lifecycle Management tool. With the introduction of support for automated sprint planning, teams reported spending less time on estimating the work / creating tasks thus spending more time with the product owner to understand the sprint priorities and requirements. It helped Scrum Masters achieve better engagement levels throughout the session by analyzing insights provided by the virtual Scrum Master. A few teams were also able to record an increase in planning efficiency, where planned estimates stayed closer to actual ones being taken historically in past similar stories.
3.4.3 Reaction from the team on the Sprint Planning Assistance
There was resistance from teams which have been taught to carry out breaking down stories into sub tasks and provide planned estimates against each task in hours using planning poker cards religiously during every sprint planning. Our coaches had to work in the background by feeding historical sprint progress information to the tool and generate the outcome. Results and time taken to generate it by the team and by the tool were compared. This activity was repeated for two sprint planning sessions followed by further coaching session to explain true objectives of sprint planning is to define the sprint goal and agree on how much stories the team can look forward to comfortably deliver with or without appetite for stretched goals. Some teams have still adopted it conservatively and don’t miss a chance when the virtual Scrum Master falls short of providing good enough task and/or task effort recommendations.
3.4.4 Assessment of Retrospective Session & Outcome
Retrospective meetings are one of the most important meetings within the Agile team since it allows the team to do “inspect and adapt”. The key aspect of retrospective meetings is to have open conversations between the team members with the goal to uncover what is working within the team and what’s not working. To make the retrospective meeting effective the discussed points should be more quantitative rather than qualitative. Some examples for qualitative points are “I feel the defects in this sprint have reduced”, “I feel we were good in updating tasks in the Agile tool”, “I think we were not very good in estimating the tasks” or “I think we achieved most of the stories in this sprint”.
This is the biggest challenge to get teams to be more quantitative rather than qualitative.
Virtual Scrum Master offers support for Retrospective sessions to support the team’s improvement journey. Virtual Scrum Master analyses processes and best practices followed by the team during the sprint and provide insights on the basis of “what went well” and “what didn’t go well” on quantitative basis.
3.4.5 How does the team use the Retrospective assistance?
The team uses the virtual Scrum Master’s Retrospective assistance to bring deep insights on how the team has performed during recently concluded sprint. As coaches we don’t advise the team to start with the insights of the Retrospective assistance at first, but with the team’s own retrospective so that the team doesn’t get biased based on it.
Once the team’s retrospective is done, we then look at the points given by the Retrospective assistance to align it back on the team’s feedback. The team then can create actions and assign them to various members.
3.4.6 Reaction from the team on the Retrospective assistance
The team loved the Retrospective assistance since it provided transparent feedback in terms of where the team stood on leading practices. It also provided a guided approach to run effective retrospective sessions where teams identify actions for improvement areas. It saved team effort in analyzing the data of the respective sprint and providing insights.
4. Challenges in adopting the Virtual Scrum Master
While the concept sounds great, the adoption of the virtual Scrum Master hasn’t been easy. There have been tough questions from the Agile teams on going against one of our core values of Agile—individuals and interactions over processes and tools! The perception is that we are trying to build virtual Scrum Master with the hidden agenda of taking over the Scrum Master role.
In terms of adopting the virtual Scrum Master, the challenges we are seeing on the ground for the Agile teams are classified into three areas—people, data and technology.
4.1 People
The physical Scrum Masters were not happy with the virtual Scrum Master due to the following factors, which made it difficult during the deployment:
- Their roles getting diluted. Currently, Scrum Masters in various teams are very transactional in behavior. They don’t act like enablers or a catalyst within the team. These Scrum Masters feel that soon their role will be diluted.
- The virtual Scrum Master forced them to think out of their comfort zones. They need to redefine their role to add value to the scrum team.
- In the initial stage of adoption, the Agile teams felt that the virtual Scrum Master required additional learning to use it effectively plus for the project team there is an additional cost of getting a coach to support them on it.
- The teams saw the virtual Scrum Master as a hindrance since they must be disciplined in following the process and provide the required data because the virtual Scrum Master requires data to be intelligent.
Teams felt that the virtual Scrum Master impeded the collaboration and discussions within the team during the scrum ceremonies. Teams felt that the virtual Scrum Master was taking the team towards command and control behavior since the virtual Scrum Master alerts are viewed by the leadership and sometimes contradicted the views given by the team.
4.2 Data
- After showcasing the virtual Scrum Master’s capabilities, during the initial 3- 4 sprints, the Teams got frustrated since the virtual Scrum Master didn’t showcase many intelligent insights. The teams needed to understand that, it will take time to effectively showcase intelligent insights. Patience was the key for the teams. The virtual Scrum Master was as good as the underlying data from the Agile tools. Data from at least a few releases was needed to produce the meaningful probability of successful prediction.
- The virtual Scrum Master was deployed within the Accenture environment. We needed to connect with Accenture hosted /customer hosted Agile project management and engineering tools. Some organizations were not ready to share their data outside their network. Absence of access to data lead to a non-functional virtual Scrum Master.
- The deployment of the virtual Scrum Master in short duration release assignments was another challenge. The time to stabilize the virtual Scrum Master took about 4 – 5 sprints to start seeing insights.
- In building the machine learning models for the virtual Scrum Master, we needed good quality and high-volume data to train the models to predict with higher accuracy.
- The virtual Scrum Master requires data inputs and correlations from various Agile planning and delivery tools, DevOps tools, testing tools, IT service management tools etc. Connectors need to build across these toolsets.
4.3 Technology
Varied AI technology stacks such as Google, Microsoft, Splunk are available to build various aspects. The challenges focus around the right architecture and the right skilled people in these technologies.
5. Where Do We Stand?
In the earlier section, we listed the challenges faced by the team implementing the virtual Scrum Master. We would like to provide some insight on which of the challenges are still a work in progress (WIP) through the below table:
Table 1: WIP challenges in implementing the virtual Scrum Master
6. Conclusion
Today’s age is of human empowerment. It’s about us designing technology that conforms itself to people, putting us firmly in control of our own fate. We collaborate with AI and machines to do our jobs better.
AI in Agile will not replace the collaboration between and within the teams but augment it to the next level where we are continuously improving. Disrupted Agile with AI is no longer a trend. It’s a reality that’s being accelerated by increased automation in the software development process, via artificial intelligence and cognitive computing. AI and machines can help us to do Agile better. Embrace your new digital co-worker!
7. Acknowledgements
First and foremost, we would like to thank the Accenture Leadership of Rajendra T Prasad and Koushik MV for their support in making virtual Scrum Master a reality. We’d also like to acknowledge the contribution of the Architects, Development, Testing and Deployment teams in the development of the virtual Scrum Master. Lastly thank you to the Jutta Eckstein for helping to shepherd us through the creation of this experience report.
REFERENCES
https://www.businesswire.com/news/home/20160411006361/en/Accenture-Launches-Intelligent-Automation-Platform-Deliver-Smarter
https://www.accenture.com/in-en/service-mywizard-intelligent-automation-platform
https://www.versionone.com/about/press-releases/versionone-releases-11th-annual-state-of-agile-report/
This Presentation has been published for information and illustrative purposes only and is not intended to serve as advice of any nature whatsoever. The information contained and the references made in this Presentation is in good faith and neither Accenture nor any its directors, agents or employees give any warranty of accuracy (whether expressed or implied), nor accepts any liability as a result of reliance upon the content including (but not limited) information, advice, statement or opinion contained in this Presentation. This Presentation also contains certain information available in public domain, created and maintained by private and public organizations. Accenture does not control or guarantee the accuracy, relevance, timelines or completeness of such information. Accenture does not warrant or solicit any kind of act or omission based on this Presentation. The Presentation is the property of Accenture and its affiliates and Accenture be the holder of the copyright or any intellectual property over the Presentation. No part of this document may be reproduced in any manner without the written permission of Accenture. Opinions expressed herein are subject to change without notice.
