The Indian subcontinent has not seen many successful Agile Transformations. In this report we share how a group of 30 Lean Agile Practitioners and thought leaders from industry came together to better understand why Agile Transformations have not achieved or sustained the results in India that we hoped for.
1. INTRODUCTION
At gatherings all around India, the talk is about different tools, techniques and agile experiences. But many of the experiences are either not from India or only partially achieved here. Agile experiences in India invariably fall short o full-fledged organization-scale self-sustaining irreversible transformations. Most are about rather short-term limited-scale team-level externally-supported “agile adoptions” or about lower-order internal efficiency gains such as faster builds, or higher code coverage, lower tech debt, or quicker deployments. No doubt these accomplishments are important from a product development standpoint, i.e. “building the product right”, as well as a product management standpoint, i.e., “building the right product”. However, in most cases, any objective assessment or quantitative demonstration of improved agility in terms of market-facing metrics (such as revenue impact, percentage of market share, number of paying customers, etc.) is conspicuously absent! Roles in functions such as product management or user-experience design have only begun to spring up in last few years. So the matter of fact is not many have seen real Agile Transformation in India, and those who have seen it, recognize that scaling it across the organization and sustaining it over a reasonable period of time is a wider and chronic problem.
So, to bring out the real state of Agile in India in the open, we (Deepti Jain and Tathagat Varma) called for a Summit—a “Change Agents Summit,” to create an authentic account of the situation.
Deepti Jain: I run an Agility Solutioning Boutique as well as a not-for-profit country wide Lean-Agile CoP by the name “AgileVirgin”. The former provides custom solutions for her client’s needs for Agility, and the latter hosts various paid as well as free events to educate the community, among which prominent are: Agile-A-Thon, AgilityToday, WomenInAgileAndTech, Change Agents Summit and FUnconf. On this path, when I realised that just doing my job right will not help and a movement to surface real challenges is required. These Challenges are faced as well as imposed by these Organisations. And so Tathagat and I decided to callout Change Agents in a Summit and create a report on same, that can be shared across the nation and help everyone on different roles and designations in Industry.
Tathagat Varma: I started my journey as an “exploratory programmer” back in college days and created a fuzzy language using LISP. Over the years, my interests changed to different domains but my spirit has remained the same – to question the status quo and examine how we could make it more relevant and useful. I am an ardent student of mother nature and currently exploring interdisciplinary approach of biomimicry and complex adaptive systems on one hand, and evolutionary and social origins of civilizations and cultures on the other hand to better understand how we could improve how humans collaborate.
2. BACKGROUND
On paper, it all looked familiar and appeared highly successful. However, given that most organizations were either IT outsourcing operations working per or alongside a client’s given process (often in a tight contract lest there be legal disputes around delivery not matching the specs or project plans), or the so-called GICs (the Global In-house Centers, i.e. the in-house delivery arm for MNCs typically headquartered overseas) working on derivative versions (as opposed to “1.0” problems) or smaller-scale problems (compared to their headquarters) or on “lights on” work, or the Startups that were predominantly in early phases of acquiring or post-product/market fit; it often stood out that the landscape looked very different from the US / West Europe (where the work flowed from, for the most part).
3. OUR STORY
We were a group of 30 Lean Agile Change Agents who have worked in different companies from Banking, financial services and insurance (BFSI), Fintech, Telecom, Consumer Internet, Enterprise, Edtech, Healthcare, Energy, Cleantech and Government. We have worked in different cities of India and also outside India. This enables us to adapt to various cultures. Our working experience ranges from 9 to 27 Years.
Top Row, L-R: Abhishek Vatsyayan, Nagini Chandramouli, Purabi Misra, Sumeet Aggarwal, Sandeep Sinha, Manish Sachdeva, Vikramjit Singh, Tathagat Verma, Srinath Ramakrishnan, Vivek Garg, Siddharth Shukla, Mahaveer Bhamasha, Subodh Kumar, Pradeep Kumar, Saurabh Sharma
Middle Row, L-R: Pawan Kharbanda, Vaibhav Sharma, Sudha Kharbanda, Surbhi, Neha Arora, Sonali Kar, Priya Sharma
Bottom Row, L-R: Gaurav Anand, Ashish Agarwal, Neeraj Jain, Rajan Julka, Deepti Jain, Vandana Sinha, Dheeraj Sukhija, Praveen Panwar
On 7 Oct 2018, 30 Lean Agile Change Agents got together to share their experiences about Agile Transformation attempts their organisations or clients made, and failure and success factors. We organized the one-day summit as a kind of “Flipped Conference”, i.e., there were no keynotes, no guru talks, no case studies, no speakers! Instead, we leaned upon the participants, aptly called the “Change Agents” to learn from each other and co-create solutions. We call this Flipped Conference “Change Agents Summit”, and it has become an annual event for us, where we work on challenges faced by Lean Change Agents in India.
4. THE SUMMIT: EXECUTION
We only provided the loose structure but content and conversations from participants were used to share ideas and learn from each other. Under Tathagat Varma’s thought leadership, we ran our summit in as follows:
Context Setting: First, Tathagat Varma explained to all Change Agents the intention of Summit and how it could be executed. Four Change Agents were asked to volunteer to facilitate 4 sub-sessions around different stages of Transformation progression.
Day Flow: The day was divided in 4 parts, loosely based on 4 major Transformation Phases (detailed later in this report) in an approximate chronological but logical manner:
- Initiating Agile Adoption
- Enabling Agile Transformation
- Scaling Agile Transformation
- Sustaining Agile Transformation
Four Change Agents volunteered to facilitate each session. Each session was facilitated following these guidelines:
Set Objective (5 min): The facilitator introduced the topic and explained the objective. This helped create guardrails around the main theme of each session lest the participants veer off tangentially, which is a real possibility in an open-ended session such as this.
Brain Writing (15 min): Individuals on each table listed “Key Challenges” and “Success Factors” for the current Transformation Phase. As opposed to the generally followed practice of brainstorming where the conversations typically get hijacked by a smaller number of more dominant participants and a few topics end up getting monopolized, brainwriting ensures that initial idea generation is much more democratic, both in terms of individuals contributing to the session and the ideas that are being generated.
Brainstorming and Prioritisation (10min): After the brainwriting session, each table got ~8-10 ideas per participants, so there could be upwards of 50+ ideas on a single table alone. While not all of them might be most relevant, practical or important, this let us capture the entire spectrum for a more thorough analysis. The next step was brainstorming where each table (4-5 participants) discussed and prioritized the list for their table.
Present top ideas (10min): Each table chose a member (by rotation) to present its Top 4-5 ideas (2 min each), with any other member pitching-in in case more clarity was required. This allowed ideas to be represented as best as possible with minimum chances of errors of translation or understanding.
Collate ideas from everyone (15min): The post-it notes from all tables were sorted and consolidated on wall charts and archived. Everyone did silent voting on the Top 3 issues.
Summarise (5 min): The facilitator summarized the key outcomes of the session.
5. OUTCOME: OBSERVATION AND REPORT
This section captures key findings from the summit organized by major transformation phases.
5.1 Initiate Agile Adoption
Agile Adoption means the very start of Agile Transformation in the organisation. Here Organisations start with understanding what is Agile. Now the problem is every organization thinks “we are different”. More “successful” organizations tend to simply disregard them claiming they are already very agile, and the old-school businesses simply declare this “agile thingy doesn’t quite apply to our business”. So, it takes a lot of humility for an organization to accept that they need to learn what agile is all about, and how can it help them. A ‘big-bang’ style of transforming a complete organization is rare here (not that it hasn’t been tried; the Salesforce case study happens to be among the most-often cited one). The majority of organizations prefer the approach of starting with pilots. But the unfortunate part is that they don’t know what to do beyond Agile/Scrum training and setting up team and project level processes. They don’t know how to scale. Many organisations re-do the whole thing multiple time and ultimately settle for the state where they are, which is – largely the same old setup with few a Scrum-like processes at the team and project level.
Major challenges for Agile Adoption are:
- Lack of vision: Leadership could not do their job of establishing their vision for Agile Transformation clearly, probably because they themselves did not know and did not have courage to come forward and say that we need to learn and change. The confusion was consistent at all organization levels for adopting Agile. The only trainings that happen focus on team and project level process change.
- Lack of Business alignment: Two things here – First, the Business still doesn’t understand nor has there been enough effort to explain to the Business what Agile means. So Business doesn’t offer time for reviews and push for changing priorities and items in backlog of Active Sprint. Second, there is poor alignment of the backlog to business goals. And the reason for that is Product Mgmt OR Leadership never enrols IT/Software units in the Business Vision. Both units still work in silos. Problems only get compounded when the wrong metrics such as “how many projects are agile” or “what is the velocity of your project” are tracked (and more so when used for rewarding them!).
- Internal Resistance: Each living system has a certain level of in-built natural resistance to external change which might otherwise be healthy in keeping unwanted foreign elements out, but could also equally kill good ideas just because they are not aligned with “how we do things over here,” or other similar short-sighted biases. These internal barriers might be structural (in terms of how reporting lines are drawn, or how existing systems or processes are wired), cultural (in terms of what are the core and shared values of the organization), very specific to key leaders or individuals, or simply a natural resistance to a new and unknown idea. Typically, leadership commitment and executive support are called out as key enablers in most such surveys, and we are not surprised these are the issues reported by our participants too.
5.2 Enable Agile Transformation
Once an agile adoption is initiated, organizations soon discover that while agile doesn’t often bring about overnight success to their businesses (much to their dismay!), it does often bring almost overnight bad news! The “bad news” is typically around how internal ways of working aren’t attuned to the agile ways, or that they need to throw out a lot of stuff in order to make some headway with agile. And when initial results from agile don’t help establish alignment with the big picture but instead are seen as some new fluffy way in which there are often carelessly-thrown statement to the effect such as “we don’t know when we will be done” or that the “team decides what to do and when to deliver”, one can mentally visualize what a senior-level executive must be thinking. It goes without saying, the success at this point largely depends on the key leaders recognising that it often “gets worse before it eventually gets better” and throwing their weight behind whatever it takes to keep moving and remove any impediments that jeopardize, or even threaten agile adoption.
Enablement of agile adoption, thus, might require making structural changes (for example, merging traditional “adversaries” such as Dev and QA into a single organization in a step towards making them work as a single team), process changes (e.g. asking product management to stop making big fat Product Requirements Documentations and instead insist on lightweight product backlogs that are then continuously updated), or growing skills and competencies (e.g., transform from being an “I-shaped” to a “T-shaped” professional), etc. It could also mean bringing in tactical operation-level changes (e.g. moving from using Gantt charts to burn down charts, etc.). In short, while the team might bring daily changes to how they work inside an agile team (such as how they improve code review practices, or how they share knowledge, etc.), they also require a matching level of changes in the environment surrounding them.
Realistically, these changes are often outside the direct control of the agile team in the trenches, and invariably require a top-down intervention to enable an ongoing agile adoption lest most of the effort of team be spent in fighting the corporate immune system and their efforts be effectively rendered less potent if not altogether negated. In this session, we discussed how effective such measures were. In a nutshell, not many organisations reach beyond adoption stage.
Here are the major challenges that we found for the second stage of Agile Transformation:
- Setting realistic expectations: Quite often, there are inflated claims about what agile can do. While many of them might be realizable in the long-term, in the short-term, it means setting realistic expectations that problems will be discovered that will require a considerable amount of time, effort and money. It would also require working with the team to help them tide over a relatively bad patch and ensure that someone in the organization has their back! At each of leadership levels, it means educating the relevant stakeholders, empowering and trusting the teams to come up with recommendations on what is the right way to move forward, and then working with them provide them the necessary level of external support in a timely manner. Additionally, it also means protecting the teams against unwarranted criticism and premature judgment and working with all stakeholders and constituencies to stay focused till the goals are accomplished.
- Providing required resources: Resources such as investment in new skills, or improved technology or engaging external consultants might be required to enable an ongoing transformation. Talking about the initiative in an employee all-hands or approving the budget for a training or a tool might appear to be low-hanging fruit and an obvious sign of executive commitment, but it is not a replacement for the leadership owning an agile transformation. It also requires establishing acceptable standards of performance, and leading from the front when things don’t go well, as they eventually will. Taking out non-performers and people who actively sabotage an agile transformation is just as important as provisioning required resources. Ownership and support are not “one and done” events but a continuous activity throughout the transformation.
- Creating safe learning spaces: Teams embarking on a new way of working would mean that there often are untested hypotheses, and there is a high chance that some (or many!) of them could fail for reasons that are not fully known, anticipated or understood beforehand. If mistakes are castigated and failures are shunned, this has a direct bearing on people’s willingness to sign-up for them. A “space” offering psychological safety would mean there is a culture that encourages people to surface concerns (even if unfounded), teams signing up for moderately risky “safe to fail” experiments, and any mistake or a failure being seen more as a learning opportunity rather than an individual’s failing and even being “rewarded” for it.
5.3 Scaling Agile Transformation
This stage was a far-fetched dream OR most difficult stage for most of the participants. Their maximum understanding was either around how it’s done via scaling frameworks (dominantly SAFe) or some kind of Scrum-of-Scrums. This was a difficult one for all. Those who are working closely on the Agile Transformation of their organisation shared that as soon as they reached this stage, the transformation initiative became completely directionless. Only a few like Phillips India or SalesForce India were proud examples where scaling was achieved via SAFe. Many participants shared that after hitting a roadblock somewhere mid-way, organisations re-did the whole thing, again and again.
Most of the approaches don’t really bring about a holistic change in the organization in the structural sense of the word. Instead, they bring about a project organization structure with its own roles and norms that help ensure large teams can work in a way that is considered more agile than the traditional approaches. While the jury is still out whether these are really “agile”, and there are several commercial interests at stake that only gloss over their shortcomings, we wanted to understand how organizations are actually scaling agile adoption across the organization notwithstanding what commercial framework they are using (just like in the first session on agile adoption, we didn’t really focus on which agile methodology they were using).
Here are major challenges and reason for total failure of this stage, in India:
- Creating a shared vision: Organizations that don’t share a vision end up being dysfunctional, and run at multiple speeds. A part of the organization that is working at high agility could soon find itself frustrated because other peers in the same organization don’t share the same perspective and instead try to maximize their own returns. Unless explicitly orchestrated by top leadership, this mindset could thwart any such local effects and not just derail scaling up and across the organization but even force the teams in question to retreat back into their respective shells. Creating a shared vision and establishing a common standard of acceptable performance, which is aligned to the shared vision, is critical to help teams and functions understand the new priorities of the organization. It is especially important to explain linkages between teams and functions across the organization to help people recognize how their work affects other teams.
- Re-aligning organisation: Structure plays a huge role in shaping the flow of work inside an organization, and eventually on the culture of an organization. In sociological terms, it also limits the “agency” an individual brings to the organization, and thus affects individual mindsets and attitudes about an organization. While too much structure (and reinforced by performance KRAs) could kill teamwork and stifle innovation, too little teamwork could confuse people about who is really responsible. It is important to re-align organization around the new shared vision that best allows the teams to operate by minimizing internal friction and maximising value delivered to the customers
- Establish consistency: One of the prerequisites of scaling up an organization is to set up common and consistent processes that allow repeatability across locations even when original creators of the process are not around. We recognize that software development is not manufacturing but more like a discovery process, and it often creates tacit knowledge that can’t be captured effectively in a straightforward process. That said, we still need to have some common flows that help orchestrate work across teams. While these would require tweaking, for every software project is different, they still retain high-level checks and balances and internal mechanisms for how critical milestones are progressed. Consistency at a certain level also requires defining roles and responsibilities, or acquiring common tools and systems, or enforcing common standards of code reviews and testing across different software teams, while respecting the fact that each software project is different. Easier said than done, no doubt this is also a leadership job!
5.4 Sustaining Agile Transformation
Scaling remains a challenge, but sustaining has become a bigger challenge. To go about sustaining a successful transformation, we first had to talk about what a ‘successful transformation’ is. Being able to meet short-term goals could be seen as a legitimate win. However, just like a patient on a life-support system might only be seen as having biological life (though even that might be questionable!) but not in the generally accepted sense of the word “alive”, an organization that still requires its army of consultants, coaches and incentives to keep it “agile” might not really qualify as being agile! Indeed, most organizations report a virtual overnight collapse of their so-called agility within months after all such external scaffolding was taken away. Just like we don’t have a “WaterfallMaster” and yet the teams gleefully continue to practice waterfall, would your scrum team continue to operate just as before (if not better) without its ScrumMaster? Major findings:
- Evolving a Culture: Transforming to an agile way of working eventually is all about evolving to a culture where individuals are free to practice their individual agency and operate from a growth mindset. It is important to call out “evolving” as culture isn’t something that can be copied, or artificially imposed. While we frequently see books and talks around how some of the organizations have built great cultures, it would be extremely foolhardy to even so much as consider copying those practices and expecting similar results. We must understand that a culture is simply the immune system of an organization that has evolved over time in order to maximize its chances of survival and sustain growth in its particular conditions. Needless to say, when external conditions change, as they eventually will, the same old culture could practically become useless. Every organization is unique in terms of its people, its values, social norms, nature of business, operating environment, etc. Therefore it makes sense for an organization to invest in evolving its own unique culture that will set a strong foundation for scaling operations as it grows, and sustaining it when the chips are down!
- Establishing Boundaries: A great culture doesn’t mean it is laissez-faire! A laissez-faire system would actually soon turn into a very chaotic system, and its polar opposite, a fascist system could soon degenerate into a highly toxic organization. The magic is in being able to right-size organizational boundaries and to allow individuals and groups to retain their unique strengths and competencies while enabling them to collaborate to create an impact larger than what is possible by a single unit. Successful organizations recognize that unless commensurate changes are made to organizational structure, there can’t ever be a long-lasting and self-sustainable transformation. Clear boundaries help people navigate effectively without stepping on each other’s toes or altogether dropping the ball.
- Enable Continuous Learning: An agile organization thrives on its ability to continuously learn and adapt from both its internal and external environment. There are three key elements to this: learning, sharing and adapting. Learning can happen to anyone, in any function, at any level and anytime! More often than not, a lot of learning happens at the touch points with the customers, and these happen to be at rather junior roles or peripheral positions! Failing to listen could mean missing key cues and losing agility in the long run. These learnings, then, need to be shared in appropriate forums for some form of dissemination, analysis and actionable guidance for the rest of the organization. Thus, having a platform where such learning could be shared becomes critical. Finally, any such learning is useless if it fails to translate into concrete actions taken on the ground. As the popular saying goes, “if nothing changes, nothing changes!” Thus, an organization must establish a culture where learning is seen as a badge of honor, never mind if it was successful or not; asking for help or sharing help generously is seen as a critical business necessity and not just an individual altruism; and finally, an action-orientation keep the flywheel running shared vision. Organizations that don’t share a vision end up being dysfunctional, and run at multiple speeds.
6. CONCLUSIONS
Agile Transformation in India is a far-fetched dream. India teams often don’t have full-scale empowerment to make decisions around a true organizational transformation. Structural and process change decisions are not made here in India. India centers are predominantly delivery centers with little bearing on a global company’s strategic plans. Moreover, they are often chartered as part of cost-containment strategy for legacy systems than for launching new innovative products or for access to newer emerging markets. While innovations were talked about in most companies, it was mostly internal in nature rather than external in terms of impact to the topline. Whatever little was happening in the name of “transformation” was predominantly within software teams at/around Director or a group manager level (and sometimes at a VP-level too). So, there wasn’t much of a business buy-in to the whole matter of agile transformation, let alone the creation of a tight linkage between organizational strategic goals and teams’ operational plans. While software in itself is fast-becoming a strategic business priority in companies’ respective headquarters, software teams in India are often limited to being on-time, on-specs, on-budget, and on-quality. Only rarely do they get “approval” to innovate or establish business-critical or market-facing goals. India leadership teams are expected to become high-performing engineering delivery centers by improving how the teams work, but not by making the “tail wag the dog”, i.e. make upward cascading changes to how marketing does their work or how HR organizes their systems. Last but not the least, the challenge of Indian organisations is being part of the overall cost-management strategy for legacy systems. Often India gets disproportionately lower investments in, say, hiring top talent (imagine hiring Ivy School talent for an IT outsourcing team, or paying top-dollars for an IIT CS student for a version 7.92 release). We won’t say it is not happening, but the majority of operations are focused on managing costs. In fact, most companies come to India due to two or more reasons: lower manpower costs (in the entry-level roles, it is still not unusual to see a 1:5 cost arbitrage, though it peters out to probably 1:3 or 1:2 and in some rare cases where there is a true global role, it might even be 1:1), higher availability of engineers (though the jury is still out whether most of them are even employable, let alone being really called out as talent!), English language skills (probably an edge over other outsourcing destinations that don’t even have this level of proficiency), and a process mindset (very critical during 90s when Indian IT industry was still coming up). Since India is not a large market for most such global companies, there isn’t much expected in terms of innovation for the local market either. Thus, up until most recently and even now only limited to top global companies, India centers are almost always execution centers but never really innovation centers.
7. NEXT STEPS
We will continue organising this summit as an annual event for gathering data points and then sharing our report across the industry. We hope to create a greater awareness of the state of Agile Transformation in India. We hope that when leaders pay attention, they think of these data points we have gathered, and do the right thing. Not Big-bang but at least one step at a time. Our next summit is planned for 19th Oct 2019, in Bangalore, India.
8. ACKNOWLEDGEMENTS
Very special thanks to Sonali Kar for unconditional support, Tathagat Varma for his thought leadership and master plan for execution of the Summit, for tremendous work on survey and report, and to Abhishek Vatsyayan, Nagini Chandramouli, Purabi Misra, Sumeet Aggarwal, Sandeep Sinha, Manish Sachdeva, Vikramjit Singh, Srinath Ramakrishnan, Vivek Garg, Siddharth Shukla, Mahaveer Bhamasha, Subodh Kumar, Pradeep Kumar, Saurabh Sharma, Pawan Kharbanda, Vaibhav Sharma, Sudha Kharbanda, Surbhi, Neha Arora, Sonali Kar, Priya Sharma, Gaurav Anand, Ashish Agarwal, Neeraj Jain, Rajan Julka, Vandana Sinha, Dheeraj Sukhija, Praveen Panwar for investing their time, and contributing authentic accounts from their experience. Last, but certainly not least, many thanks to Rebecca Wirfs-Brock for not giving up on me, and taking me to completion on this report. This paper would not have come together without Rebecca’s keen insights, questions, and edits. Thanks so much Rebecca, it was impossible to do it without you!
