RESOURCES

Yahoo! Global Homepage Sprint 50 and Counting

About this Publication

Abstract: Yahoo Global Homepage is web’s leading homepage destination with over 200 million users and 3.09 billion page views. This paper provides an insight into how homepage achieved an optimized release process to deliver highest quality of production ready code in every 3 weeks.

HOMEPAGE EVOLUTION AND CHALLENGES

Homepage sprints began in mid 2008. Growing in complexity and with a sheer volume of 200 M users, ad-­‐hoc change requests started pouring in from over 38 international markets. This resulted in product instability, long sprint cycles of 37 days, increased time to markets and dip in team morale. These were the primary challenges when I took on the program management ownership of Homepage.

Unlike other teams I had been on before, the Yahoo! Homepage team did have very fixed dates -­‐ special events for coverage. This required the team to trade off scope and occasionally quality rather than schedule and resources, which were usually fairly rigid constraints. Due to this intense release schedule, the high demands of the product, and the many dependencies on other teams, we revisited our existing release process and challenges associated with it and come up with few optimizations. This paper examines how we optimized Homepage’s release process in a phased approach with high focus on better predictability, change management with minimal influx and higher code quality.

 

 I. REVISIT PLANNING PHASE (FOCUS ON PRIORITIES AND EFFECTIVE PLANNING)

    It used to take us 10 working days from features prioritization review to sprint planning.
    We introduced the following changes:

  • Earlier product backlogs are structured to have a single consistent stack ranking with a business justification and market impact.
  • Pre-­‐planning in smaller groups (agile feature groups) to unfold internal and external dependencies along with sizing and low level estimates
  • Sprint planning meetings are held with an end goal in mind i.e. actual sprint commits which needs to be communicated to external global teams
  • Paired feature teams (product, development and QE) to have better context and control on features right from the early cycles of development in order to minimize engineering costs

End result of these optimizations was a significant 70% cut down on time taken from feature prioritization review to sprint planning i.e. from 10 day to 3 days

 

 II. ITERATIONS AND ITERATIONS TO FULLY BAKED FEATURES

Development cycles were entirely linear with a turn around time of 15 days and little early feedback from quality engineering. This was a major blocker to accommodate market requests
 Daily check-­‐ins on committed features
 Automated daily builds for QE to verify and provide early feedback
 Integrated daily performance checks to identify issues early in the cycle
 Features demonstrated to owners for sign off to ensure there are no gaps in the expected behavior

End result was a significant 60% cut down on iterative cycles i.e. from 15 days to 9 days

III. INTRODUCED POST ITERATION STABILITY PHASES

Earlier development cycles lacked the dedicated stability phase. Given the complexity of homepage, we have created feature stability phase (4 days) and product stability phase (3 days) to ensure better product quality.
 Features gaps are fixed on a priority basis to avoid huge backlog defects
 Current feature defects and legacy defects are fixed on a priority basis
 Framework to identify performance issues early in the cycle (performance checks runs on a daily basis)
 Optimized QE regression cycles across OS, browsers, buckets

Future plans to cut down stability cycle by 40 -­‐ 50% to be more agile.

 

IV. OUR BIG DAY – PRODUCTION DEPLOYMENT

Earlier Production deployment used to take us 3 days with silo’ed responsibilities.
We introduced shared management of production deployment across distributed teams using a follow-­‐the-­‐sun model.
 Content Management System production deployment is followed by the back end production pushes a day later
 Proposal to push CMS and backend pushes with a 12 hours lag
 Flip for simple markets followed by global markets
Production deployment cut down by 50% i.e. from 72 hrs to 36 hrs

 

 V. NEW INITIATIVES – INNOVATION AND BUG WARS!

We have introduced following new initiatives during our sprint stability cycles:
• Innovation day to empower engineers to contribute new product
ideas which goes through a defined process on a monthly basis with an expected outcome of mapping the selected ideas to sprint cycles.
• Bug wars on outstanding defects to have a better control on product
quality once in 3 sprints.

 

VI. CONCLUSION

Earlier development models contained overlap across 3 sprints. We have eliminated overlapping significantly. With sprint 50 and beyond, we plan to further optimize our release to make it truly agile as part of Yahoo!'s 2011 goals set by the PLT