In The Pocket is a digital product studio, creating products on requests of clients. At first we had a small team and the products had limited budgets. Gradually, our team grew and the products became more business critical. Today we have six teams each creating multiple products. To build products that make users happy and businesses grow, our process evolved to a combination of design thinking, lean and agile. As these theories are often described from a product team perspective, this gives rise to challenges in our case. We overcome these by forming teams with a technology focus, expanding some agile roles and keeping sprint meetings relevant. On top, budget control is often neglected in the above theories. Because of its importance, we have introduced regular budget control. This results in teams enabled to build and grow multiple products, within the agreed budget.
1. Introduction
Agile literature often describes a team working on one single product. However, in reality, this is often not the case, leading to challenges with respect to focus and information sharing. If that is a recognizable situation, this report provides some practices to deal with this reality. On top of that, budget control is often left out of the equation. We describe how we agree on a budget and monitor it.
In The Pocket creates digital products for a multitude of clients, with a focus on value for budget. A couple of years ago, we had one small team creating products on fixed budgets. Now, we have evolved to six teams, each managing several products with larger budgets in different stages of the product lifecycle. These teams are autonomous and fully cross-functional, with the integration of a product manager and a product designer in each team. Our company’s mission is to build products that make users happy and businesses grow. To fulfil this, our process evolved to a combination of design thinking, lean and agile.
As these theories are often formulated from a single product perspective within a single company, this gives rise to some problems: Who is the product owner? How is information shared? How is the team focusing on the specific tasks at hand? Who controls the budget? We have overcome these challenges by expanding some agile roles, forming teams around a technology focus, allowing sprint meetings to have a single focus and giving teams room to innovate. The available budget is controlled in recurring meetings, where the added value is always checked against the delivered effort. All this is described in greater detail in this experience report.
As a result, our current process and team structure enable the teams to focus on building great products and to regularly improve their performance. It furthermore allows the company to scale, while maintaining the same service level towards our clients.
The next section gives some background on In The Pocket and depicts the company organization. Next, in section 3, our process, which is aimed at delivering value, is described in more detail. After that, in section 4, the challenges introduced above are elaborated with a presentation of the applied solutions. Section 5 focuses on the performance analysis of the teams. At the end of the report, some concluding remarks are presented.
2. Background
In The Pocket is a digital product studio. 85 strategists, designers, engineers and support staff work together in our headquarters in Ghent, Belgium. Our mission is “to create digital products that grow companies and make people happy”. In The Pocket sets up strategic partnerships with companies willing to invest in their digital future. Together we depart from a product strategy to make the products grow, far after the initial launch.
These are some product highlights we’ve released recently: a payment app for Bancontact, the market leader of electronic payments in Belgium, a shopping list buddy for Colruyt, a well-known Belgian retailer, and Stievie, an OTT platform for Medialaan, the largest commercial broadcaster in the country.
The cornerstones of In The Pocket are autonomous, cross-functional teams, capable of conceiving and delivering products for our clients. At the moment, we’ve got 6 teams in place. These consist of multiple engineers, a QA engineer, a solution architect, a product designer, a product manager and a team lead. This is depicted in Figure 1. Every two teams have a dedicated account manager, and all teams are supported by strategists, talent, office, IT, sales and management. The teams all have a certain technology focus, e.g. web or react native or mixed reality. To enable learning and development within a certain area, we have horizontal competencies, from web or iOS development to product strategy.
Figure 1. Team Structure
This experience report is written from the perspective of the Agile Coach from In The Pocket. It’s his job to make sure everyone in the company understands and applies the principles that are in place, in order to maximize the value that the teams deliver. In addition, he helps clients to follow the company process and assists them in their agile transformation, if needed.
3. How we work
3.1 Combining design thinking, agile and lean
Before building a product, we want to make sure that we understand the problem our client wants to address. To this end, we apply design thinking (1), where a repeated pattern of diverging and converging is applied to 3 steps. First, the user needs are understood and defined, e.g. by focusing on jobs to be done or by making an experience map indicating pains and gains in the current user journey. This is done with the support of our strategists, who have a clear view of the client market, new trends and technologies.
When the needs are clear, different solutions are explored. Methods used in this step are a team-wide brainstorm or design studio session, where in a short time multiple approaches are iterated on for a well-defined challenge. Finally, the solution decided upon is worked out in more detail, in mockups or a prototype. We try to validate this prototype with users as soon as possible, to incorporate valuable feedback as early as possible. All these steps happen within the team.
Once it’s clear what solution to build, the team starts developing the product in an agile way, following the well-known principles. Currently, all teams adopt the Scrum methodology with 2-week sprints, with the regular sprint meetings taking place (2). Even though the process of discovering what to build and then building it may seem like 2 distinct steps, they are interwoven and happen continuously, where the output from the discovery phase is input for the delivery phase. This is what is commonly known as dual track agile in today’s literature (3). It is visually presented in Figure 2.
As mentioned in the introduction, our process also has a strong lean focus: we’re not building all features requested by the client without a critical evaluation, instead, we aim at a minimal viable product that can be launched as soon as possible. Besides that we set goals driven by actionable metrics. Through validated learning, we can validate the assumptions we’ve made in the previous steps. Our tagline is ‘deploy or die’ and we’re continuously living up to this and convincing our clients to launch sooner rather than later, e.g. in beta-mode. By launching, we get the most valuable feedback on the product and are able to grow it further. This also enables us to work towards the goals and KPIs that we’ve set together with the client. This is elaborated in section 3.2, on delivering value.
To summarize, with design thinking, we want to explore enough options to come to the right solution. By applying Scrum we want to build this solution as efficient as possible. The lean mindset makes sure we start small and viable and learn by validation. As we’re agile in everything we do, in the inspect and adapt sense, this is continuously refined and optimized. Some challenges we are currently facing are described in section 4.
Figure 2. Dual Track Agile
The process described in this section is the result of continuous learning, refinements and applying theory to our situation. Five years ago, there was one large team and the agile working was quite naïve, with a full upfront design and backlog creation. Scrum was not applied yet at that time. This shifted with the creation of two dedicated development teams and a product team. The development teams began working in sprints and following the scrum guidelines. The product team began mastering design thinking to come up with the best solution for the problem at hand. Finally, last year the teams became fully cross-functional, enabling them to autonomously build and grow products without external dependencies. As the products grew in complexity and became more business critical over that same time, the lean approach came into play, with a focus on MVP thinking and setting goals to check whether the product is indeed living up to its expectations.
3.2 Delivering value
As described above, we start with discovering the user needs, to come up with an optimal solution. At the same time, when starting a new product track, we also build insight into our client’s goals & which KPIs they have in place to evaluate the outcome of the product. If these are not set yet, we define them together. In the end, there is agreement on the goals our solution should serve and by which KPIs this will be monitored.
Next to the KPIs, a set of strategic themes is defined to focus on. Every 6 months, these themes are re-evaluated. The proposed product features are evaluated according to these strategic themes, to check how they serve in reaching the KPI target. As it is easier to prioritize a limited set of themes, compared to a multitude of features, this helps in defining and limiting the first release. This allows creating a roadmap based on goals, instead of features (4).
During development of this first release, we focus on which analytics to include, in order to have as much valuable data once the product is launched. With this data, we monitor the KPIs together with our client, to check whether certain themes should be further invested in, or are considered done, or should be discarded or changed. This agrees with the pivot or perseveres strategy described in The Lean Startup (5). By focusing on the outcome of a release, we try to deliver as much value as possible.
As an example, for the shopping list buddy mentioned above in section 2, first the experience of managing the shopping list was optimized, like adding items on a variety of options and sorting the list based on the store layout or product category, before introducing other features, such as letting the user indicate his allergy and lifestyle preferences or sharing the list. This decision was based on user validation and analytics.
4. Multi-product teams: solutions to the challenges
This section dives deeper into the challenges introduced by having multi-product teams and describes the solutions that are in place to overcome them. The first subsection addresses the agile roles and client involvement. Next, the lack of single focus is dealt with. After that, the budget monitoring that is in place is highlighted. Finally, the mechanism to cope with idle time is described.
4.1 Agile roles and client involvement
As you can see in our team structure above, the standard agile roles of product owner and scrum master are not present. This is not because we see these as not important, but because in our team setup, some other tasks and skills are required. The first difference is a team leader instead of a scrum master. In our case, the team leader is responsible for the delivery towards our clients and for the people management of the team, e.g. the performance and personal development of individual team members. On top of that, he can take up scrum master responsibilities or these can be divided over the team.
A second difference is the absence of a product owner, while each team has a product manager. This is because we have split up the product roles: the role of (agile) product owner is taken up by the client, where the product manager in the team is a broader expert in translating needs to solutions that can be built. The product manager oversees the discovery phase and maintains the backlog, together with the product owner at the client side. This last one sets the priorities and both product manager and team lead co-manage the client stakeholders. As such, they both are a proxy for the product owner at the client and support or coach this person.
The dual-track agile process described above demands a high involvement of our clients: determining what to build, getting internal validation, refining the backlog & attending sprint demos. For some, this can be overwhelming. We try to overcome this with a clear onboarding when a new client joins us. By explaining what we do and why we do it, we make it easier to align our activities with our clients and identify the right stakeholders to interact with.
4.2 No single focus
As mentioned in the introduction, we have 6 teams building products for a multitude of clients. At the moment, 5 teams are building more than 1 product, while only 1 team is working on a single product. These products are typically in different product life cycle stages, some are being defined, others are close to a launch and certain ones are operational and in a grow phase. As our teams are technology focused, all products within this technology are taken up by the same team, while under contract. From an agile perspective, this poses some challenges, as most literature focuses on a product team with a focus on one single product. As a consequence, in our case, not the whole team is simultaneously working on the same product, while it is beneficial that all members are involved.
This involvement is facilitated by the technology focus of the team. This enables team members to follow up on pull requests, even if they are not actively developing the product. To keep sprint meetings productive, they can be aligned with the clients. For example, a stand-up can be held for client A today, while client B is discussed tomorrow. Backlog refinements and sprint reviews are done with the client, so separately for the different products in a team. To keep everyone on the team informed about each other’s activities, the teams are encouraged to hold a show and tell with the entire team, to inform all members about each other progress.
To have an optimal information sharing, we strongly believe in the 3 amigos concept, where stories are discussed by product, development and QA members of the team. This can be done both in the product concept phase, as when a story is taken up in development.
In a true agile spirit, our teams always are optimizing their performance. This continuous optimization is done with regular retrospectives. To allow for all aspects to be improved and because the teams are cross-functional, the retrospectives are focused on the team working, or on the technical quality of the team’s code, or on the product integration within the team. A multitude of retrospective formats is shared within the company, like the speedboat (what is slowing you down or speeding you up), start-stop-keep doing formats, looking back on the sprint with tweets, the moving motivators (finding out what motivates the team members), a lean coffee retro, superheroes (what superhero are you, what power & weakness, what nemesis & sidekick) and many more.
4.3 Budget agreement and monitoring
We are a digital product studio, also known as an agency: we build digital products for clients, within their budget. We’ve evolved to beyond budgeting principles: the client buys a certain team size, within which we maximize the output, with the process described in section 3. This poses 2 challenges: procurement is often not ready for the framework we propose. Secondly, when a certain budget is agreed upon, this needs to be monitored.
To overcome the first challenge, we try different approaches: we’re working on agile contracts where procurement of large companies can agree on, by focusing on the value that will be delivered, as described in section 3.2. Another approach is to agree on a first budget to launch an MVP and follow up from there. If this still is a too risky for a client, or too time consuming to close a deal, we start with a dedicated discovery track, to refine the budget further when there is a clear view and agreement on what to build.
At frequent intervals, typically after two or three sprints, we have an alignment meeting with the main stakeholders, including the budget owner to monitor the progress. All available information is discussed: the current progress, an outlook based on the previous team velocity and upfront high level or detailed estimates. On top, the product strategy and ambitions for the upcoming period are aligned.
The need to increase or decrease the team size to accommodate the foreseen planning or budget is discussed. This meeting is governed by the account manager, with the assistance of the team lead and product manager. The account manager oversees the overall budget and makes sure that continuation is guaranteed, in order to reach the goals that are set for the product. This often requires the allocation of extra budget.
By using the techniques described here, we have long-lasting partnerships with about 2 out of 3 clients.
4.4 Idle time & innovation
As we rely on client business, it can happen that the workload of certain teams is lower than others at some points in time. In this case, we’re not changing teams to maximize staff occupancy, but rather we encourage to work on company strategic initiatives or products initiated by the teams themselves. As a basic measure, a team can invest 7% of its time on its own products and way of working, and 17% on company-wide products and initiatives. This includes the horizontal competencies to allow learning and development within specific domains.
These 2 buckets allow for valuable work even in case of no client work and are a driver to innovate and experiment with new technologies. In the long run, this will also benefit our clients. As this is a driver for innovation, we also encourage teams to invest in their team time when there is enough client work. For all our internal initiatives and products, we follow the same process as described above.
An example of such a product is our first VR application, matching wine with its origin and food that pairs well. This allowed us to gain experience in this new domain and have a product to showcase the opportunities of mixed reality to our clients and prospects.
5. Performance analysis
The approaches described in this experience report are all aimed to allow the team to work as efficient as possible, within their specific situation. The checks and balances we have in place to monitor the team performance are outlined in this section.
First of all, we’re monitoring the committed versus achieved velocity, where we’re aiming at a ratio of 90%. This is to drive a realistic planning and allow for a proper expectation management towards our clients. As a team is typically working on different products, this can be a challenge. Each sprint, only one team in three reach the target ratio. As such, this is a continuous focus for all teams. A dedicated planning per product helps in getting a small gap between committed versus achieved velocity.
As velocity is a lagging metric that is only available after a sprint, we’re also focusing on cycle time. This focus implies that dependencies are signalled quickly and taken up swiftly. Internal dependencies are taken up immediately as the team is fully cross-functional. External ones are signalled and followed up intensively. All teams use electronic scrum boards, where dependencies are flagged and always accompanied by a comment reflecting the latest status. Stories have a cycle time of about three days, on a median basis. Bugs typically have a shorter cycle time. Having technology focused teams, allows them to have a consistent flow.
Besides these basic metrics, each month an agile health survey is filled in by all teams. This is a lightweight survey, inspired by the Spotify team health check (6), to determine how the teams evaluate themselves on certain agile aspects: proudness, the speed of work, team ambiance, the challenge of work, sprint meetings and learning. This information allows us to see company-wide trends and is served back to the teams, to use during their retrospectives to improve their own working. In one team, at the start of the year, the team ambiance was significantly lower compared to other teams. This was caused by recent team switches and the introduction of a new challenging product. By feeding this information back to team, both causes were addressed and the score for the team ambiance increased over time.
Generally, all teams score high on productive sprint meetings, which is the result of a sustained focus on the importance of each dedicated meeting and reflects that the approach to split up meetings according to clients helps. Next to that, team ambiance also gets high scores, except for the specific case described above. This illustrates that, although a team hosts multiple products, people enjoy working in the respective teams. Learning within the teams improved since the start of the surveys, as the time available for it has become transparent to the teams and they are encouraged to take up this time. Proudness on the delivered outcome and challenge of the performed work has mixed scores. This indicates that working on multiple products often in different product stages and with a different inherent complexity, remains a challenge. Therefore, the practices written in this report are not the end game but need continuous attention and coaching effort.
6. Concluding remarks
As intuitive as our process might seem, it is the results of continuously optimizing how we work, with a focus on the value we deliver to our clients. Combining design thinking with agile development is nowadays gaining popularity as dual track agile. Furthermore, we have a lean mindset, with early launches and learning from what you can measure. By focusing on goals and strategic themes, priorities can be set, maintained or altered easily and with all available information, including the most important: how users are interacting with the product.
The challenges that arise from having multiple products in a team are overcome as best as possible. This is a mixture of what works for a small subteam focusing on one product and for the whole team, with the same technology focus. As all products and clients are different, there is no optimal solution for this. Rather, we try to work according to the agile principles: a strong interaction with our clients and continuous delivery and improvements. Although all teams now follow the Scrum methodology, it can well be that some change to Kanban in the near future, if this enables them to deliver value more easily.
Agreeing on an extensive budget in an agile context remains a big hurdle for large companies. Once there is agreement on a budget, we have mechanisms in place to monitor both budget and outcome at regular intervals. As we hold the value we deliver high, we set up KPIs together with our clients and we build and follow goal based roadmaps. This allows to further grow the products we build and our clients’ businesses.
7. Acknowledgements
I’d like to thank In The Pocket, for always improving its way of working and the opportunity to present it in this experience report. Furthermore, the contributions of Hannes Van de Velde and Christophe Rosseel are highly appreciated in improving this manuscript. Finally, I want to thank Eduardo Guerra, who supervised this report as shepherd, and provided valuable feedback.
REFERENCES
- Tim Brown, 2009, Change by Design, HarperCollins Publishers, New York
- Jeff Sutherland, 2014, Scrum: the art of doing twice as much in half the amount of time, Crown Business, New York
- Introducing Dual Track Agile – The Theory, https://medium.com/@emabolo/introducing-dual-track-agile-27a23d12268b, published 2016, accessed April 2018
- No more features on product roadmaps – Have themes or goals instead!, https://medium.com/@maa1/no-more-features-on-product-roadmaps-have-themes-or-goals-instead-484bde68a990, published 2015, accessed April 2018
- Eric Ries, 2011, The Lean Startup, Crown Business, New York
- Squad Health Check model – visualizing what to improve, https://labs.spotify.com/2014/09/16/squad-health-check-model, published 2014, accessed April 2018
