Agile strategies are being applied on more-and-more data warehouse (DW)/business intelligence (BI) projects within government agencies than ever before. The heavy, serial, long-running traditional approaches of yesteryear are being cautiously abandoned for the lighter, collaborative, incremental approaches resulting from agile and lean thinking. But how does this work in practice? Agile strategies call into question virtually every single practice that the data community holds dear, and are often at odds with existing IT governance strategies, so how are agencies reconciling this in practice?
This presentation overviews both an agile and a lean strategy for teams developing a DW/BI solution. These strategies are based on a tailoring of the Disciplined Agile Delivery (DAD) and Data Vault 2.0 (DV2) frameworks. We begin with a review of the differences between the traditional and agile ways of thinking, what the implications are for your organization, and how this can impact government agencies in the process of moving towards more agile ways of working. We then overview two lifecycles for DW/BI development, a Scrum-based agile strategy for the first release of a DW/BI solution and a lean Kanban-based strategy for future releases. Strategies for successfully initiating an agile DW/BI team will be covered, including how to weave user stories, agile data modelling, agile architectural modelling, and light-weight release planning together. A collection of agile database techniques will be overviewed, including agile data modelling, database testing, database refactoring, continuous database integration, and DV2 design strategies. The presentation ends with a summary of the people, process, and tooling challenges that you will face when transitioning to an agile approach on your DW/BI teams.