The Covid pandemic with its unprecedented lockdowns and border closures has caused global business disruptions that most of us haven’t witnessed ever before. One of the biggest challenges for many of us Agile leaders has been adapting our human centric, collaborative ways of working to the remote working compulsions imposed by the pandemic. This experience report captures my learning that agile product delivery can still work, if we stay true to the principles and leverage collaboration tools to the fullest.
1. INTRODUCTION
In early 2021, Thoughtworks was engaged by one of our financial services clients for a complete revamp of their current rewards program. The client provides a cloud based financial services platform with a variety of products for the financial operations of all types of small and medium businesses (restaurants, laundromats, hairdressers etc.). These businesses rarely bought the client’s products directly, instead they did so based on the referral of their accountants or bookkeepers whom they relied on for financial advice.
To incentivise accounting and bookkeeping practices (called “partners” to recognise their role in growing the adoption of the client’s products) to refer their products more, the client has a rewards program which rewards the practices with increasing commissions, discounts and service levels as they refer more and more of the client’s products.
In the last few years, the referral sales had been stagnating, leading the client to question the effectiveness of the current rewards program. The goal of the engagement was to understand the shortcomings of the current program, and together build a new revitalised program that would restore the partners’ confidence again. Some of the key challenges identified in the current program are captured in Figure 1.
| The program is complex and messy, with many different ways to earn points | Referral sales are low and dropping | Partners demonstrate low awareness and uptake of benefits | Bookkeepers show lower engagement in the program than accountants |
Figure 1. Challenges in the partner rewards program at the start of the engagement
We heavily leverage on the lean startup principles in our agile product development engagements to ensure that the client gets the most valuable product for the money spent. We follow the build-measure-learn loop, where we co-create small increments of the product with our client, measure the business value delivered against our original hypothesis and then plan the next increment based on our learnings.
This approach requires a great level of collaboration both with the client as well as end customers, which during normal times, is achieved by fully or partially co-located delivery. The pandemic however brought with it a unique set of challenges to the execution of this engagement.
2. Background
Typically, every Thoughtworks product delivery engagement (whether onsite or distributed) begins with an intensive 2-4 week discovery and inception, during which a small team runs collaborative, workshop style activities to gain a better understanding of the business context, validate the problem space and arrive at a solution approach working closely with client stakeholders. The stress is always on lo fi tools and techniques like co-location, flip charts, white boarding and post it notes to encourage a lot of ideas, discussion, debate, and collaboration. Figure 2 below shows a typical Thoughtworks discovery/inception and the lo fi outputs.
Figure 2. Collaboration and outputs in a typical ThoughtWorks inception
Subsequently some periodic in-person interactions are also built in that can be exercised based on the ongoing collaboration needs.
3. Our Story
The client has been engaged with Thoughtworks in their home region for a few years now, but mainly in co-located product delivery or in offshore maintenance/support engagements. The existing working relationship was smooth, and there was consensus within Thoughtworks that the client is really aligned with our agile product delivery methodology.
The decision to revamp the existing rewards program was finalised a few months before the pandemic hit, and funding was allocated assuming they could build upon the existing relationship and access he pricing benefits from distributing to our lower cost regions. From the Thoughtworks side, we felt confident that we could execute this as an offshore engagement, so long as some team members could travel for the initial discovery and inception, and then on a smaller scale at quarterly intervals going forward. We also felt reassured that with only a five-hour difference in time zones, we would still have about a half work day worth of overlap between the team in Thailand and the client stakeholders.
With these assumptions in mind, we recommended a fully remote engagement structure with travel factored in for the initial discovery and subsequent quarterly co-location. Given that cost was a key constraint for the client, and we were trying to build up Thailand as a full-fledged delivery centre, we recommended delivering this engagement out of the Bangkok office.
As the commercial agreements were signed and we started working on the engagement logistics, Covid started circulating around the globe. We delayed the start a few weeks hoping things would settle down, but things instead began to deteriorate. Both the client’s home region and Thailand reacted to this situation by closing borders and enforcing among the world’s strictest lockdowns.
This sudden global travel freeze created several challenges for us in the execution of this engagement.
3.1 Problems
The pandemic introduced some significant challenges in how we would staff and execute such an engagement:
- Running a remote discovery/inception
Given this was a complete revamp of the existing rewards program, it was critical to have a well-executed product discovery and inception. Identifying the key pain points and drivers for the business, building the hypotheses to address them, and building a product roadmap with the right priorities required a lot of face time, white boarding and discussions together. This was especially true given the disconnect on the program direction among the key client stakeholders.
Given co-location was not a possibility, we had to figure out how to recreate the same experience and get the same outcomes digitally.
- Ramping up on the business domain quickly
Thoughtworks works on the model of encouraging consultants to be generalists i.e., we do not typically encourage specialisation in business domains or technologies. The team in Thailand had a limited understanding of the financial domain in general and even less so about the client’s home country. In the normal times, we would have augmented this with some local help pairing with Thai colleagues during the inception.
The pandemic has made remote working the norm, therefore from a commercial perspective, it was very difficult to convince the client to pay much higher rates for letting us bring in some local Thoughtworkers to bridge this gap. This made the learning curve for some of the business contexts much steeper.
- Testing hypotheses and designs with users before implementation
The client had informed us in the very beginning that the typical accountants and bookkeepers at accounting practices are not very tech savvy. Therefore, user research for the proposed product hypotheses required focus group sessions and 1 on 1 meetings, with a lot of handholding, to get useful feedback.
We initially planned for this to be covered by in person focus groups during discovery and then user interviews every quarter. With the restrictions on travel, we had to figure out ways to conduct research online that would help us effectively reach this user base.
- Measurement of delivered value after each release had to be swifter, and any changes had to be implemented quickly to match up to competition
In a business refresh of this magnitude, it is inevitable that client stakeholders have very different perspectives on what to do, when to do it and how to do it. In person interactions go a long way in resolving disagreements between key stakeholders and in helping arrive at a consensus decision. This is especially true at the start of the product development where the level of divergence is the greatest. Unfortunately, the cancellation of the co-located discovery meant we lost the forum to build initial consensus on the program direction.
3.2 What we did
By bringing our diverse experiences together, our team worked through these challenges:
Validating hypotheses and prioritising product features
Given the difference in time zones, we were finding it really challenging to get access to all senior stakeholders for a joint session. For a few sessions that we managed, we found the participation and engagement level to be really spotty.
To overcome this, we used Miro to collaborate asynchronously, capture pain points and opportunities in the current rewards program from all stakeholders and prioritise them. We then asked them to map the hypotheses against the top five. Finally, we ranked them based on the customer need (Desirability), effort and complexity of the technical solution (Feasibility) and estimated business value (Viability). Figure 3 shows an example of features ranked by DVF criteria:
| Feature hypothesis | Pain Point Addressed | Desirability | Viability | Feasibility |
| Offer self service capability to understand loyalty tier, benefits, expiration etc. | Partners demonstrate low awareness and uptake of benefits | High | High | High |
| Earn loyalty points for completing educational courses on the products | Partners demonstrate low awareness and uptake of benefits | High | High | Medium |
| Offer self-service to improve visibility of earned benefits and ease of use | Partners demonstrate low awareness and uptake of benefits | High | High | Medium |
| Average points earned by size of the practice to level the playing field | Most referrals come from larger practices and very few from 1-3 member practices | High | Medium | Medium |
| Offer specific course content for bookkeepers | Bookkeepers show a lower level of engagement than accountants | Medium | Medium | Low |
Figure 3. Ranking feature hypotheses by DVF criteria
Breaking the feature hypotheses into a list of stories
Once we had the feature prioritisation done, the next step was to break down the priority features into story maps. The story maps helped us collaboratively build the component list of storyies required to implement the feature. The story maps also helped us visualise any missing pieces, sequence stories in the right build order and prioritise only the ones that were an absolute must for the feature to work. Figure 4 below shows an example snippet from the story map.
Figure 4. Building a visual story map from prioritised features
Building the user experience for the new program
The next step was to start building a compelling user experience for the proposed program. Again, we found that starting from an empty canvas would not be too productive in a remote working setup. Therefore, we decided to leverage Figma for iterating through the design. We went through several iterations – first very lo-fi sketches of the various options, then increased fidelity mockups on the narrowed down options followed by a high-fidelity mockup of the finalised version before the user interviews. Figure 5 below shows this evolutionary journey which we achieved using Figma.
Figure 5. Evolution of the user experience from ideas to hi fidelity mockups
Conducting user research online
Once there was broad agreement among the internal stakeholders on the user experience, the next step was to get customer feedback on the proposed designs. Given that most accountants and bookkeepers were not very tech savvy, we needed to make user testing intuitive and very close to the in-person experience.
Rather than walking them through a bunch of mockups, we used Maze to ask them to perform specific tasks all the while observing them and getting their feedback as to what they expected. Figure 6 below shows an example user testing session facilitated by our UX designer Numan.
Figure 6. Mimicking in person user testing using virtual tools
Leveraging data analytics for quick decision making
In a fully remote setup where getting face time is a huge challenge, analytics made gaining stakeholder consensus a lot easier as the real numbers always won over personal opinions or viewpoints.
For example, after the first release, there was a push from a section of stakeholders to give higher weightage to referral sales from smaller sized practices to level the playing field with larger practices. However, when we looked at the data, we found that almost 99% of practices on average referred once a year. Therefore, it would antagonise the few practices that contributed the bulk of referral sales income if we tried to level the playing field. After looking at this data, the leadership agreed to reward points based on absolute numbers v/s numbers weighted by practice size.
| Referral sales count ranges | Count of practices referring in the range | Total referrals in the range | Average referrals in the range |
| 0 – 99 | 24, 124 | 35, 526 | 1 |
| 100 – 199 | 42 | 5, 490 | 131 |
| 200 – 299 | 5 | 1, 109 | 222 |
| 300 – 399 | 7 | 2, 456 | 351 |
| 500 – 599 | 1 | 558 | 558 |
| 600 – 699 | 1 | 675 | 675 |
| 800 – 899 | 1 | 866 | 866 |
| 1200 – 1299 | 1 | 1, 295 | 1, 295 |
| Grand Total | 24, 182 | 47, 975 | 2 |
Figure 7. Analytics data showing the distribution of average number of referral sales
3.3 Results
After 18 months, I feel these are the highlights and key achievements in our journey so far.
- Running virtual wireframing sessions successfully
We’ve been able to deliver multiple releases of user facing changes in the rewards program in the past one year. We’ve also built a great cadence of introducing incremental changes in the user experience with each release and getting them approved quickly. We’ve identified a set of internal client stakeholders and customers who are highly knowledgeable in the accounting/bookkeeping domain and very interested in the evolution of the product. We bounce off the initial designs with them, and firm up the high fidelity designs based on their inputs. Doing this ensures we have a mostly solid design before going to approval from the larger group of stakeholders and user research. This allows the development team to start working on the feature earlier, as the designs at most need minor tweaks based on user research findings.
- Built an environment of trust in working with remote team members and clients
We have been able to transition to an entirely Thailand based team. We started by leveraging resources from our Singapore office for their product development experience and English language proficiency. Over time we were able to build confidence in both the Thailand development team as well as the client stakeholders to understand each other and build a smooth and trusting work relationship. In fact, the client has remarked how that the transitions felt almost seamless to them.
- Releasing working software frequently
An important requirement for the client was to be able to do incremental improvements in the loyalty program on a regular basis. By building an effective and lightweight feedback loop, we were able to build features, release them quickly for feedback and incorporate this feedback into the following release.
- Building the right thing
Despite the difficulties in collaboration introduced by the pandemic, by leveraging the virtual tools and some changes in our ways of working, we were able to ensure that we always built and released the features with the highest business value and impact for the end customers.
4. What We Learned
Below are some of my key learnings from this experience:
1. When co-location is not possible, a lot more preparation is needed to make virtual collaboration just as effective.
When people are physically together, collaboration is a lot easier and can also be a lot of fun. We may go through a lot of flip charts and post-its, but it is natural and intuitive to brainstorm and design together in person. In remote settings on the other hand, we found that rather than starting from scratch, it is more effective to prep work within the team first, and then walk through these options with the larger group of stakeholders. Trying to replicate in-person interactions in remote settings quickly leads to meeting fatigue among the participants.
2. There are a lot of virtual tools out there and it is easy to quickly get overwhelmed. As a team, we need to identify the leanest set to help us meet our collaboration goals.
The pandemic has led to a proliferation of virtual collaboration tools, both paid and free, for our various project needs – meetings, whiteboarding, story mapping, sprint planning etc. Each of them has its own advantages and disadvantages, and it is very easy to get lost in the plethora of available options.
The best approach would be to start with a leanest set that meets the team’s needs, and then add more when we find a gap. Some helpful criteria in selecting the right toolset for us included compatibility with the client’s security setup, familiarity within the team members, cost, ease of installation and use, and lastly but most importantly the closest match to the collaboration needs.
3. While the execution might have changed, the underlying principles of Agile product development still hold water and are highly effective.
Building and releasing a small increment of the product, measuring value of the released increment and using the measurement to drive the next increment are the key tenets of the Agile product development approach. The need to release quickly and frequently keeps us focused on doing just enough and speeds up decision making. The need to measure value helps us build something that is objectively and transparently measurable. Tying the next increment to the measured value and feedback ensures there is less disagreement on what the next feature is. All these pieces working together really help in a remote working setup where we don’t have the luxury of multiple discussions to drive consensus.
4. Despite our best efforts, remote working can never be successful without the continued and patient support of our clients in adapting to the new normal.
Adapting to the new normal was a learning adventure for us, but it was also the same for our client. This transition would not have been possible without their Information Security and Tech Operations teams working with us. They supported us through the journey by rapidly onboarding our Thoughtworks laptops to access their networks, giving us access to internal systems like Jira and Tableau, and approving new collaboration software for use on an urgent basis. change?
5. Acknowledgements
I’d like to thank my colleagues Numan and Beena for their ongoing support; I learned so much from you two! I’d also like to thank my employer ThoughtWorks for this enriching opportunity.
Last, but certainly not least, I’d like to thank my shepherd Siva for his pragmatic advice and guidance to help me shape this paper.
REFERENCES
Ries, Eric “The Lean Startup” Crown Publishing Group, 2011
