Top 10 Tips for Measuring Agile Success

Measuring Agile Success

Choosing the right agile metric to measure agile success is really simple, right? I wish that were the case, but in reality, selecting the correct agile metric can be a little tricky.

So, how do you get the most out of your agile metrics? I reviewed the 9th annual State of Agile survey, which compiles insights from nearly 4,000 respondents, to find out how agile practitioners are measuring the success of their agile initiatives.

#1 On-Time Delivery

According to the State of Agile survey, 58% of the respondents* said they measured the success of their agile initiatives by on-time delivery.

With agile, our schedule is fixed and our scope is flexed. What does that mean for on-time? Well, time just happens, so theoretically, we are always on time. But, on-time is generally measured in context with the expectations about what will be delivered. To measure and have visibility of what is being delivered, we may look to the out-of-the-box metrics of the burndown or the burnup.

For instance, in this VersionOne burndown chart, you can see progress as the team heads toward an expected end date.

Agile Metrics 1

This burnup chart, on the other hand, allows you to see the trend of getting stuff done, as well as the impact of scope changes.

Agile metrics 2

#2 Product Quality

A total of 48% of the respondents to the survey said they measured the success of their agile initiatives through product quality.

Quality is often measured in multiple ways, including looking at customer satisfaction, revenue growth, and the technical aspects of testing conducted throughout the development life cycle. With agile software development teams, we’ll look at our velocity of completing working software with quality built in. We tightly couple continuous testing and inspection throughout the lifecycle of the development, so we’ll constantly be monitoring testing trends as well as constantly inspecting build and code health.

For instance, in this testing trend chart you can see the cumulative progress around testing activities. Ultimately you want to see all green, but a large amount of red along the line might reflect some issues in the code base or process.

Agile metrics 3

#3 Customer/User Satisfaction

The survey found that 44% of respondents measured the success of their agile initiatives by customer or user satisfaction.

As with all these benefits, there are multiple ways to measure the outcomes. In the case of customer/user satisfaction, these include looking at the Net Promoter score, sales figures, number of support calls vs. number of features delivered in a time period, or usage statistics of product or site capabilities.

#4 Business Value

Approximately 44% of the respondents to the State of Agile survey stated that they measured the success of their agile initiatives by business value.

And several of the principles of the Agile Manifesto recognize the importance of delivering business value. Measuring business value is very explicit when we know that there’s a contract for work to complete or a compliance need and fines if we don’t finish the work. On the other hand, sometimes measuring value is prospective or speculative in the sense that the market inputs drive decisions and the value is often a best guess. Having a business value score applied to the features to be delivered can measure value.

Here’s a sample epic cumulative flow chart based on value. This helps you see the delivery of anticipated business value as features and other large stories are completed. 

Agile metrics 4

#5 Product Scope (Features, Requirements)

Another 39% of the respondents answered that they measured the success of their agile initiatives with product scope.

Setting a goal around what to get done over the next three months, then tracking status, and getting it completed is hugely rewarding. Actually having real-time feedback as to the progress of work is valuable to everyone on the team, from the engineers to the program managers. With agile software development projects, you can always rely on the burndown charts, or just visualize the progress of the cards moving from left to right on the project kanban board.

Here’s an epic board in VersionOne that helps the team track and visualize the progress of features at a program level. Or if you’re using Scaled Agile Framework® (SAFe®), you can see progress at the release train level.

Agile metrics 5

#6 Project Visibility

Project visibility was the measure of choice for 30% of respondents to the survey.

One of the best ways to build trust is transparency. That means having the plans out in the open and making progress visible to all. Sharing progress at multiple dimensions provides the different stakeholders with information that makes sense from their point of view. Metrics that show featured or overall progress against a targeted plan can provide great insights.

In this chart, you can visualize progress on a feature and known work. The red diamond represents what was actually anticipated, so it’s easy to gauge whether you are above or under the anticipated scope.

Agile metrics 6

The other reason visibility is important is that we need to have alignment among internal teams so they can best manage their work in relation to component or service dependencies.

Understanding the impact of one team’s work on another team is critical. By looking at the dependency chart below, it’s easy to identify the stories at risk.

Agile metrics 7

#7 Productivity

According to the State of Agile survey, 29% of the respondents said they measured the success of their agile initiatives through productivity.

The concept of productivity in an agile world is a measure of outcomes, not output. So looking at burnup for a product or based on value is hugely impactful. Simply looking at a burnup of count of stories or features over time is a great way to understand how much the team is actually delivering.

#8 Predictability

Approximately 25% of the respondents from the survey said they measured the success of their agile initiatives by predictability.

A predominant metric used to assess predictability is the velocity trend. For a three- to four-month period, this shows how much work has been completed at a sustainable pace on average. A wildly fluctuating velocity might reflect a team that is changing, work that is unpredictable, or simply a team that is still getting used to defining work small enough to complete in an iteration.

A velocity trend chart like the one below not only helps you see the performance but also gives you visibility into whether or not the team’s output is at a predictable state – as this one shows.

Agile metrics 8

You can always try to assess velocity based simply on the count of story cards completed every week. This is usually the best indicator of predictability.

#9 Process Improvement

Another 23% of the respondents said they measured the success of their agile initiatives by process improvement.

A core tenet of all lean and agile mindsets is continuous improvement – constantly getting better. But how do you know you are getting better unless you are measuring the outcomes? There are all the metrics above that help, but there’s also the extremely helpful cumulative flow chart which shows how well work is flowing through the lifecycle.

With this team-level cumulative flow chart, you can see where bottlenecks or slowdowns may exist.

Agile metrics 9

Also, there’s cycle time – which helps us with planning and predictability. Cycle time is a great metric to view over time to see if process tweaks and adjustments are having an impact on productivity.

For instance, in this cycle time report, you can see the level of variability and performance across the various estimated pieces of work.

Agile metrics 10

#10 Don’t Know

Just 11% of the State of Agile survey respondents said they didn’t know! Well, if you don’t know the benefits, try to start looking at the metrics above. You’ll see improvements in delivered value, better quality around what is produced, a more predictable cadence, and ultimately happier customers.

Conclusion

These results show that there isn’t just a single metric that everyone uses. Different organizations, types of management, and teams need different metrics.

Not sure which metric is right for you? Check out the 9th annual State of Agile survey to learn more about what nearly 4,000 of your peers are doing.

What metric do you use to measure your agile initiatives’ success?

*Respondents were able to make multiple selections.

State of Agile is a trademark of VersionOne, Inc.

This is an Agile Alliance community blog post. Opinions represented are personal and belong solely to the author. They may not represent the opinion or policy of Agile Alliance.

Add to Bookmarks Remove Bookmark
Add to Bookmarks Remove from Bookmarks
Add to Bookmarks Remove from Bookmarks
Matt Badgley

Matt Badgley

Matt Badgley is a Lean/Agile Coach and Product Consultant at VersionOne. He is a Certified ScrumMaster (CSM), Certified Scrum Product Owner (CSPO), and SAFe Program Consultant (SPC). Matt has worn one too many hats in roles from Systems Analyst to Programmer to IT Manager to Programmer to Director. He’s a seeker of new ideas and learning new things. His passion has been working with product delivery teams to develop into their own identity and simply…

Recent Blog Posts

Recent Posts

Your Bookmarks

No favorites to display. You must have cookies enabled to add bookmarks.

Recent Agile Alliance Blog Posts

Ensuring safe, inclusive Agile events
The mission of Agile Alliance is to support people and organizations who explore, apply, and expand Agile values, principles, and practices. When members of the Agile community have concerns about our pursuit of that mission, it is important to address those issues.
Ensuring safe, inclusive Agile events
The mission of Agile Alliance is to support people and organizations who explore, apply, and expand Agile values, principles, and practices. When members of the Agile community have concerns about our pursuit of that mission, it is important to address those issues.

Post your comments or questions

Discover the many benefits of membership

Your membership enables Agile Alliance to offer a wealth of first-rate resources, present renowned international events, support global community groups, and more — all geared toward helping Agile practitioners reach their full potential and deliver innovative, Agile solutions.

Not yet a member? Sign up now