In a world of editable genomes and customizable cancer treatment, in order to keep pace with rapidly advancing research it’s time to reconsider how the academic lab operates. The Broad Institute of MIT and Harvard is one of the world’s leading genomic research centers, where more than thirty academic laboratories, groups, and platforms work collaboratively to enhance human health and medicine. Kendra West, co-founder of the Broad Institute’s affinity group “Agile Academia”, has coached many Broad Institute laboratories and team members on incorporating Agile values into their day-to-day workflows and operations. This report details the Agile journeys of multiple Broad Institute laboratories and introduces challenges faced by the teams — including competing priorities, staggered timelines, scientists’ skepticism, and how best to support inherently individualized work.
1. INTRODUCTION
One of the most impactful roles I’ve ever had was studying viral infectious disease as a research associate in an academic laboratory. This lab was part of the Broad Institute, one of the world’s largest genomic research institutions, and my group specifically sequenced samples from viral outbreaks and used the information in order to track and fight disease. It’s a fascinating process. By sequencing (or “reading”) hundreds of patient samples in an outbreak scenario, there are tiny mutations, (or “typos”) which can indicate how the virus was spread from person to person. Viruses spread and mutate quickly, so if two people have the same “typo” it’s likely that they came in contact at some point and the virus was transmitted. Not only can the spread of a virus be detected in this way; sequence information can also be used to identify potential genetic locations for diagnosing and treating the disease. I joined the lab shortly after the 2014 Ebola outbreak in West Africa and many of my new colleagues had dedicated their past year supporting the outbreak response efforts of our African partners. A few months after I joined the lab I travelled to Nigeria and helped to set up a sequencing center at a large university in a town called Ede. I was blown away by the incredible compassion and urgency with which the laboratory worked; I truly felt like we were changing the world.
Before joining the infectious disease team, I worked with the Broad Institute’s Genomics Platform, a large-scale sequencing operation capable of processing thousands of genetic samples every day. This was my first introduction to genetic sequencing and I quickly found myself fascinated by the process and application of the technology. This space is also where I first encountered active process improvement; the platform incorporated Lean manufacturing principles into the laboratory through dynamic workflow design (Dodge, 2016). The group spent years reflecting upon and optimizing their process in order to reduce costs while increasing processing speed and quality. This was my first exposure to active process enhancement, and before I knew it I was hooked on finding ways to make work easier and more efficient for all. After joining the infectious disease laboratory, I was keen on finding ways that I could use those same optimization techniques to enhance my new team’s amazing efforts. I began to look for changes the group could make to decrease our response time and allow us to operate more smoothly and at a higher efficiency. We were working so fast, and our mission was so urgent, that there were many things slowing the lab down that just weren’t a priority to fix. I was working with some of the most brilliant minds in science but those brilliant minds were working in multiple directions. If better aligned, it felt like we could be working more effectively for a greater impact. It was while considering continuous improvement in the academic laboratory that I discovered Agile.
I was introduced to my first Agile Coach while trying to develop a digital workboard for my laboratory team. She worked for the Broad Institute’s software development platform, and her enthusiasm and experience coaching Agile teams quickly convinced me that this was something I needed to learn. I had a strong desire to apply Agile-based tools to the lab, but didn’t yet have the experience or “toolbox” to get me there. In order to better learn and understand Agile, I started a role as a scrum master with the Broad Institute’s Data Sciences Platform; a methods development and software engineering platform dedicated to maximizing the impact of the data sciences on the life sciences (Broad Institute, 2018). Since starting this role I have continued to work with several other people, laboratories, and departments around the Broad Institute on how to adapt Agile-based tools to enhance their laboratory or team environments. This report details what I’ve learned and describes Broad Institute laboratories’ Agile experiments with enhancing communication, transparency, and continuous improvement.
2. Background
2.1 The Broad institute of MIT and Harvard
The laboratories and teams detailed in this report are located at the Broad Institute of MIT and Harvard; a nonprofit biomedical and genomic research institution located in the heart of Kendall Square in Cambridge, Massachusetts. Since its establishment in 2004, Broad Institute scientists have published over 8,500 scientific research publications to propel the understanding and treatment of disease. The Institute’s goals span widely, from assembling a complete picture of the molecular components of life, to discovering new therapeutics or treatments for major infectious diseases, to unlocking discoveries hidden within complex genomic data sets (Broad Institute, 2018). Broad’s 1,300+ employees work in over thirty laboratories, programs, and platforms; including a cutting-edge software development platform and one of the largest sequencing operations in the world. The Institute prides itself on being a cross-functional organization and was built to include members of a diverse array of scientific fields to better foster interdisciplinary collaborations. The Broad is trying to change the way that the world does science, and operates on values of global collaboration and inclusion, knowledge and data sharing, and empowering scientists to accelerate our understanding of disease. Broad’s values are Agile unto themselves, and much is done to encourage Broad employees to follow a path to maximum scientific impact through micro grants and the flexibility to try new technologies and collaborations.
2.2 An Introduction to the Academic Laboratory
In order to best grasp the challenges for Agile practices in the academic setting, it is important to understand how academic laboratories generally operate and function. Members of an academic laboratory perform hypothesis-based research projects within the laboratory’s field of study. An academic laboratory is run by and named for a Principal Investigator- a “chief scientist” who establishes the lab’s research focus and scientific direction. This scientist has a PhD in their field and many years of research experience. Experiments are often defined by students and lab members, but the lab’s Principal Investigator (PI) has the final say in the direction and scope of the research. Most labs are small, but lab sizes can range from 1-40+ members. The roles of these lab members vary. Undergraduate and graduate level students can work with the lab on their PhD or Master’s thesis project, generally on a short-term, multi-month rotation, or for a longer multi-year affiliation depending on the project. Another role in the lab is a post-doctoral fellow. These are individuals who have recently completed a PhD and are working to gain additional research experience before starting their own laboratories or entering another advanced research position. On average, post-docs stay with a lab for 4-5 years (NIH, 2012). Other members of the laboratory include staff scientists and research technicians, full-time individuals working in the lab in a professional versus academic capacity. Larger laboratories can also have additional support staff such as lab managers, project managers, grant managers, and administrative assistants, among others. Funding for laboratories is generally grant-based; academic laboratories do not specifically work to produce a profit. The “currency” and legacy of a lab come in the form of high-quality results and publishing in well known scientific journals.
There are several organizational factors which make the academic environment a challenge for the application of Agile-based techniques. There is the challenge of project structure. As mentioned above, academic laboratories have numerous lab members who often work on entirely different projects. This is a challenge in multiple ways as projects are numerous and can be difficult to track. Many projects means many different priorities. A graduate student’s thesis project is likely to be their number one goal, while a staff scientist may focus more on a technical development project or the latest laboratory collaboration. These competing projects and priorities can mean redundant experiments and siloed knowledge. Academic laboratories most often run on grant funding, and the accidental duplication of an experiment can mean thousands of dollars unnecessarily spent. The high turnover rate of students and post-docs can mean that there is also a high turnover of knowledge within an academic organization. If a graduate student develops a new technique and priority is not placed on sharing or documenting their work, that same technique can leave the lab with said student because no other lab members know how to replicate their methods. This turnover also offers a challenge in terms of group organization. If a knowledge-sharing system or standards are not established- one student’s valuable experimental samples can end up buried for years in the back of a freezer. As leader of the lab, the PI often acts as mentor and supervisor for all lab members. Between shepherding students through their education, and overseeing the additional research and collaborations happening in the lab, the PI generally does not have capacity to focus on operational optimizations. Lab managers can assist with these types of tasks, but it remains difficult for this one person to orchestrate the needs and desires of many different people and priorities. A laboratories’ inherent limited funding also makes it difficult to hire a person to focus on organization or process, or to pay competitively for an expert to help in this area.
2.3 Agile at the Broad Institute
While the academic lab has its inherent challenges, these laboratories remain some of the most innovative and cutting-edge research operations working to advance science. In recognition of this, in Fall of 2016 Broad’s Agile Coach and I co-founded “AgileAcademia”- Broad Institute’s first and only Agile affinity group which focuses on enhancing and improving academic team function through the application of the Agile practices. This group meets monthly to discuss a team’s challenges, or a concept or tool from an Agile framework. Since we started a year and a half ago the group has grown from eight to over eighty members and has representation from twenty-two Broad Institute teams, spanning from the laboratory to finance and administration. Individuals of all roles and backgrounds attend these events in an effort to learn ways they can enhance their teams. Last year the group co-sponsored a Scrum certification course open to all Broad employees interested in learning more about Agile-based techniques. Our educational and discussion-based events remain well attended and the group continues to evolve as more Broad Institute laboratories and departments experiment with embodying the Agile values. I have been fortunate to meet many inspirational teams throughout the development of this affinity group. This report highlights their Agile journeys as laboratory teams continue to work towards enhanced communication, transparency, and continuous improvement for maximum scientific impact.
3. Overcoming Challenges
Given the uniqueness of each laboratory, teams at the Broad Institute have encountered different challenges while incorporating Agile-based techniques. In true Agile form, these teams inspect and adapt their processes in order to overcome the obstacles at hand. Generally, the implementation is pushed forward by one or two shepherds within the lab; individuals who see the benefit of incorporating the Agile values into their teams. These individuals have been introduced to Agile mainly through AgileAcademia at the Broad Institute or through project management or professional education. I have worked with several teams at the Broad on adopting Agile-based techniques, but in many of the following examples, it was others who championed their teams’ Agile implementation. I’ve met with these individuals to hear about what they have done in their groups, and to help generate next steps to try with their teams. The following are challenges encountered through mine and my colleagues’ experience.
3.1 Challenge 1: How to start
It can be intimidating for teams to jump right into Agile practices. So much of scientific research is iterating off of another’s experience, and there is a general absence of other Agile case studies in the academic field. Without the “proven example” of a laboratory using standups or retrospectives, it is hard for academic teams to imagine how it can work for them. Focusing on the benefit of teams adopting the Agile values is what was truly transformative.
To emphasize this benefit, it is important that Agile is introduced in an accessible manner. For one laboratory, their lab manager and I gave an overview of Agile to present the values and various Agile frameworks and tools. We opted to present examples, first, of how Agile works in software, and second, of the Agile tools that could be of use for an academic team. Even during the presentation there were many hesitations: “But we don’t have a product owner, how would this work for us?” or “This sounds like a lot of meetings, how will we get our long experiments done?”. The team was wrapped up in the “what” and “how” of the frameworks versus the “why” behind the practices themselves. In future presentations with other groups, I culled many of the software specifics, and approached introducing the frameworks and tools based on the values they embody and the characteristics that they work to enhance.
Other Broad laboratories found implementation success in avoiding potentially intimidating Agile terminology. Instead of a “retrospective” a lab would have a “project reflection”, or instead of a “standup” a team would have a “check-in.” Regardless of what these meetings and events are called, they still serve the same purpose: to remove barriers so that the team can move faster.
3.2 Challenge 2: Team Structure (or lack thereof)
It is difficult to focus on team improvement with no immediately identifiable “teams” and when each person’s work could have little to nothing in common with another’s. Different groups remedy this in different ways. For one lab, the “team” is centered around the whole laboratory. In this case, a group of nearly 30 people participate in the laboratory retrospective to improve the full laboratory team. Once a topic for improvement is identified, lab members who do not wish to discuss the matter can use the “law of two feet” and leave the retrospective meeting. In doing this, they acknowledge that decisions will be made without them, and that they are comfortable with the consensus their peers will come to. As a result of this rule, the self-assembled sub-team of peer stakeholders varies from retro-to-retro, but it is always centered around an improvement for the larger laboratory “team.”
In other cases, laboratories find success creating smaller community of practice (CoP) groups based on an individual’s role. One lab created CoP focus groups within the laboratory for students, post-docs, and staff scientists. These CoP groups meet every month to discuss their work, challenges, and obstacles to their success. Other people in these meetings include the Principal Investigator and at times additional laboratory administrative staff. Aligning individuals by their role in the laboratory versus the focus of their work offers a way for the group to improve that role’s function and experience. Take, for example, a student focus group. While a single student’s ultimate priority may be with their personal thesis project, gathering all students as a CoP group allows attention on accelerating or removing obstacles from the success of all of the students as a whole, ultimately accelerating each student’s individual project and exposing the students to additional perspectives on their work. These focus groups allow for enhanced communication to move the group towards inspection, adaptation, and continuous improvement of their laboratory processes. Additionally, communities of practice are a highly effective way for the laboratory’s PI to hear from all members of the group at one time. While laboratory members will still need guidance specifically around their individual projects, these CoP meetings are a convenient way for the PI to address other types of important group-wide conversations.
3.3 Challenge 3: The lengthy research process
While software is a fairly fast moving and agile industry, laboratory work can often take a much longer time. For example, genomic sequencing can take up to seven days from start to finish for processing and sequencing of a sample. Other experiments can run for multiple days, if not weeks or months. When Agile-based techniques are applied to the lab, they need to be adjusted to fit each group’s different work cadence. Standups are a tool that many Broad Institute groups have adopted for their teams. Laboratories that run longer process-type protocols found that a daily standup was not only difficult to schedule around lab work, but that they were also not an effective use of time. With a daily standup, many individual’s day-to-day updates or potential blockers did not change due to the long runtimes of their lab experiments. These teams worked to adjust the cadence of their meetings to a lower frequency, generally to a bi-weekly or once-weekly cadence. Other teams group their Agile meetings in shorter-term bursts around the needs of a particular research project. For example, one week a group could meet daily while a shorter term experiment is being done, and the next week the same group could meet only once or twice while the experiment is being run in a longer, multi-day process.
3.4 Challenge 4: Tracking multiple, complex projects
A common complaint from the laboratories is that individuals aren’t connected to the work done by others in the lab; that they don’t know what’s happening in all areas of the lab. Many laboratories find success enhancing team communication and transparency using visual workboards, especially in more process-based labs where many team members are working on the same type of workflow. With so many projects or samples in flight at once, tracking work in a common location offers an easy way for lab members to stay in tune with what’s happening around them. The challenge here is to figure out a good system for tracking this diverse array of complex laboratory projects. It’s difficult to recommend one method or type of board when each laboratory works so differently. In one group’s case, they began with a digital work tracking system but quickly found it to be too cumbersome for their needs; it was taking too long to enter and update all of their samples for each step of their lab process. This particular group found success in transitioning to a physical board located right in their workspace. They further simplified their tracking method by moving samples in batches versus as individual units. This board still highlighted any bottlenecks or holdups in the team’s process, but it was a much simpler system to observe and maintain while also having a closer proximity to the team and higher level of observation.
Another lab began tracking work in a collaborative spreadsheet, but quickly realized that it did not have the granularity needed to offer a true “pulse” on how everyone’s work was progressing. This group worked with the Broad Institute’s IT department to build out a digital workboard with many connecting pages, workflows, and ticket hierarchies. The resulting work tracking system was extremely complex, but also extremely useful for all team members. This intricate board could tell earlier in the process when one person’s work might intersect with another lab member and allow them time for better coordination and planning between the two individuals. In an effort to keep the system up to date, in a retrospective meeting the lab team requested the purchase of dedicated laptop computers for inside the laboratory space so they would not have to use their personal computers. This way they could still access their sample tickets and workflows on a computer while avoiding the time and risk of cleaning their personal computers for use in the decontaminated laboratory space. Taking the time to organize experiments into a visual workflow enhanced communication between laboratory members and offered transparency on where everyone’s experiment was in their process. This ultimately identified impediments and sped up the entire team.
4. What I Learned
The academic laboratory is by no means a straightforward space in which to apply the Agile values. That said, laboratories at the Broad Institute have still explored ways to foster high performing teams by experimenting with these frameworks. In true Agile form, the key factor in each groups’ success has been to inspect and adapt. Just like no two software teams are exactly alike; no two laboratories are exactly alike. Different tools and techniques must be used to complement each space. Laboratory work is very different from software development, but the same intentions behind the Agile values can be applied in this environment. With a bit of imagination, it is easy to see how the Agile manifesto for software development could be modified to complement and enhance this unique laboratory space. Some lines remain unchanged, and some have been adapted to better represent the academic setting. This is a first iteration. As work continues with the laboratories the ideas will evolve. An Agile Manifesto for the academic lab could state:
Individuals and interactions over processes and tools
Seeking improvements over sustaining practices
Collaboration over competition
Responding to change over following a plan
To dive in deeper, “Individuals and interactions over processes and tools.” This first statement remains the same. It is more valuable and effective for a scientist to speak with peers in the lab about a challenge than to struggle alone through documented processes or tools. To do their best work, laboratories should encourage collaborative problem solving to share knowledge and foster relationships. While it could be easy for a laboratory supervisor to hand a student a book on “How to design a great experiment and write a thesis”, having specific conversations around the student’s questions or needs will yield a far more useful set of information for them.
The second line, “Seeking improvements over sustaining practices” is a reminder for continuous improvement in the laboratory. Because of the high rate of turnover, techniques are passed from person to person and often processes and procedures in the lab are “done how they’ve always been done” without understanding the reasons behind them. There is value in sustaining practices, but it’s also important for the group to inspect the status quo in case it can be further improved. Academic teams should take full advantage of this high rate of turnover for generating new ideas. In software development, the second line from the Agile manifesto is “Working software over comprehensive documentation.” This does not work in the case of a laboratory, because comprehensive documentation is a key factor in tracking experiments. It’s critical that an experiment can be replicated by other parties; a “working experiment” that cannot be reproduced is of no value.
The third line of the academic lab manifesto states, “Collaboration over competition.” This line echoes the collaboration aspect from the third line of the software development Agile manifesto (“Customer collaboration over contract negotiation”) but has been adapted because the original is not a perfect fit. “Collaboration over competition” better captures the reality of the laboratory setting. There’s a certain amount of competition useful in a lab to work with a sense of urgency and publish big findings quickly. There are also times in the early stages of research when something could be published but groups wish to further explore the finding before others learn about it and begin to do their own explorations. While waiting to publish could be great for gaining recognition of that one specific laboratory, publishing results early for others to use will ultimately foster more rapid discoveries for the whole field. For example, when my former infectious disease lab first sequences a new virus, they often move to publish the sequence right away in order for the information to be available to other researchers sooner. They could keep this information private and work to discover whatever they can before sharing knowledge, but this would come at the expense of other potential contributions to the cause.
The final line of this new manifesto is, “Responding to change over following a plan.” This remains unchanged from the manifesto for Agile Software Development; a large part of the meaning behind this line is inherent to the scientific process. In the academic realm, a scientist must design a thorough experiment to ensure breadth and coverage of a specific question. That said, results indicating a significantly different hypothesis than expected (i.e. a “change”) could be more valuable to pursue than the original experiment. It’s useful to respond to change to drive research in a progressive and productive direction, even if it’s not the direction that was initially intended.
Agile practices have proven time and again that they’re extraordinarily effective at fostering high performing teams and organizations. Just because an industry has not applied Agile tools and frameworks, does not mean that Agile tools and frameworks cannot be useful there. With a little creativity and patience, and by focusing on the Agile values themselves, any industry can benefit from experimenting with the these practices. Broad Institute laboratories are catching on to this idea, and through open minds and experimentation these teams are crafting a new collaborative model for groundbreaking scientific research.
5. Acknowledgements
First and foremost, thank you Didi (Diolinda) Vaz for your continuous support. I am so grateful for your patience, guidance, and mentorship; without you my Agile adventure would never have started. Thank you to Candase Hokanson for your “Agile Shepherding” assistance in writing this report. I’m glad I had you here to help me along! Thank you, Professor Partha Ghosh, for your endless encouragement around my leadership development. You and the Tufts University staff have truly made me believe that I my ideas can impact the world. Thank you Jenny (Yihan) Liu, Sarah Johnson, Josh Araya, Bruce Kozuma, and Yenarae Lee, for letting me into your laboratory lives and for your openness to explore the Agile frameworks with your teams. Your experiences provide amazing examples for AgileAcademia, for other Broadies, and ultimately for future generations of scientists. Finally, thank you so much, Sarah Winnicki, Dr. Pardis Sabeti, and the entire Sabeti Lab for allowing me to work closely with you on exploring the Agile values and techniques. I am fiercely proud to have been a part of the team, and I’m driven by your work and mission in outbreak response to continue exploring how Agile ideas around communication, transparency, and continuous improvement can further enhance the way that academic laboratories do their amazing work.
REFERENCES
Broad Institute of MIT and Harvard, https://www.broadinstitute.org, 2018
Dodge, Sheila. “Using Dynamic Work Design to Help Cure Cancer (and other diseases)”. http://shiftgear.work/. January, 2016
National Institutes of Health, “Postdoctoral Researchers—Facts, Trends, and Gaps”. https://nexus.od.nih.gov/. June, 2012
