Our university curriculum development team, like many teams elsewhere, was informed that the size of the team would remain the same while the workload would increase. The ability of Agile methods to provide higher quality and productivity in software was well understood, and we began applying these methods to our team outside the software domain. We experienced common challenges working across academic departments and achieving consensus for a shared process. As expected, the power of Agile-based approaches to organizational change did not diminish the challenge of achieving a significant mindset change for the leadership. This experience report provides insights for managers working to change existing approaches in a non-software environment using the agile mindset.
1. INTRODUCTION
This is a story of how good processes can be destroyed when mindsets aren’t developed, let alone sustained. It was 2010 in Arizona where there was an event about Scrum being used outside of the software environment that exposed us to a better way for developing curriculum outside of what we’ve always known, ADDIE, which is a curriculum design model very comparable to software’s waterfall model.
2. BACKGROUND
As a self-contained team of three, Scott and myself along with a copy editor, were responsible for the flow of all online curriculum development at a university in Ohio. After some re-configuration of the Scrum framework for our needs, we successfully applied the concepts of Scrum for our own process of creating online university courses to be offered in an accelerated environment for non-traditional learners, as discussed in a previous experience report published for Agile 2011 [Wil]. In the process of applying the Scrum framework, a flow-based kanban system emerged. The iterative feedback of Scrum, combined with the visualization and flow of Kanban, brought our team of three together in a tightknit understanding of what we each needed as co-workers. Our average time for course development was cut in half from an average of 20 hours of instructional design work per course to 10 hours of instructional design work per course. Also, the consistency and communication for each sprint left the Subject Matter Experts (SME) who were contracted on a course by course basis very comfortable with the expectations we outlined for them within the Scrum framework. Our production doubled and slack became available for enhancement projects. For the three of us (director, instructional designer, copy editor), the method we had applied worked efficiently and the Agile mindset of continuous improvement and incremental delivery became intrinsic.
As the organization grew, we entered 2014 and our little autonomous self-sustaining online education team broke apart, now obsolete in the face of growth of students, adjunct faculty, and programs. Instead of online education being separate from onsite education within the college, our workload was divided into degree programs responsible for all modalities of learning. Up to this point the online curriculum team had been a well-oiled cog that was easy to be taken for granted. The new appearance of degree program departments led to Scott becoming a Program Director, and a whole new area called Academic Operations, of which I, Marian, became the director. Two support teams that represented all of the college regardless of delivery system, Curriculum and Faculty Services, developed underneath me and our copyright editor transitioned to the job of the Program Manager of the Curriculum team.
Now we were spread across the organization with our promotions, and the challenge became how to guide and influence others to adopt what we knew worked so well.
Figure 1: New Academic Hierarchy Structure within the College
…and here is where the fairytale became a predictable miasma of transition challenges while attempting to create positive, sustainable change.
3. GROWTH AND CHANGE
As the Director of Academic Operations, I sat in my office waiting for Scott, now one of the degree Program Directors and formerly the instructional designer on the “old team,” to present to me his new idea for curriculum development. There was no doubt in my mind that his idea would be interesting, because we had worked together since 2010 with a mindset that aligned with the agile manifesto. Not disappointed, Scott’s idea was that we could actually use the roles as defined by Scrum in our new organization. Up to this point, the iterative feedback and prototyping process were really the only elements we had used from Scrum due to the nature of our team and the service we provided.
However, at this point we were facing some challenges:
- The communication and logistics between the operations team that handled the curriculum development and the program directors were suffering.
- SMEs were more easily confused with more stakeholders in the pool, and there was no single point of understanding about the different roles involved.
- While the curriculum team would develop the curriculum, it was the responsibility of the program director to influence the direction of this development. However, those program director roles did not exist when the original process was developed, so the value of what they provided was unclear and thus was perceived as unnecessary complexity.
- The political challenges of getting multiple program directors to adopt the same process would not be easy due to the different ways each program director wanted to interact with that process, as well as the issue that Scott and I were peers internal to the organization. Our role was shifted from a stance of leadership and decision making in the original structure to an influencing role in the new structure for adopting an agile mindset.
- Scott and I both maintained consulting services in addition to our director roles at a mid-sized university in Ohio, so our experiences in working as consultants gave us the advantage of not being personally over-invested while having the disadvantage of not being taken very seriously by the “home team.”
3.1 Our new curriculum development structure
Given all of these challenges, the idea of assigning specific roles based on a proven method was very much needed. I had been so woven into the fabric of our original method that Scott had to remind me that we hacked the Scrum method in the first place. So we could and should do so again. It was still a bit of a hack since the team aspect is always changing in our world, but using roles analogous to Scrum ones were an essential ingredient to our new process. Our program directors were analogous to the Product Owner and the program manager was analogous to the Scrummaster. The team, with our new organizational structure, would now be the instructional designer, copy editor, and multiple writers. The shift from a single SME, or writer, to multiple writers for developing a course is enough to strike terror in any higher education administrator’s heart. However, the application of the Scrum framework with iterative planning and delivery made multiple writers not only possible, but also provided a much more robust resulting curriculum. As you can see from Figure 2, the roles analogous to Product Owner and Program Manager handle multiple short-term development teams put together for a single course at a time.
Figure 2: New Curriculum Development Structure
Scott and I beta tested this new organization structure for a single course development and team of three writers in July 2014. Scott acted as the Program Director (product owner) and I acted as the Instructional Designer (team lead). We had tremendous success. Six weeks with 1 hour working meetings were anticipated for each week, and 2-3 hours were set aside for collaboration between the Program Manager (scrummaster), Program Director (product owner), and Instructional Designer (team lead). Amazingly, the iterating, team-based process decreased development time by another 50% over our previous process. Before 2010, a course could easily take 20 administrative hours; that number had been cut in half to 10 administrative hours with the implementation of an augmented Scrum cycle process, as reported in the Agile 2011 experience report. With this new process and role presented in Figure 2, we cut that administrative cost in half again, with no compromise to the quality of the course. Rather, the course quality was incredibly strong, and we even had enough slack to spend more time involved in creative collaboration with the writers.
Three things were going well for us:
- Scott and I were deeply familiar with the approach.
- The SMEs were handpicked based on expertise and enthusiasm for change.
- The Agile mindset was already present throughout the development.
3.2 The Agile Mindset for Curriculum Design
There were two major aspects that the Agile mindset represented to us for the curriculum design process. The first, was to ensure that the course could be delivered to a classroom as a whole course at the end of each sprint. The second, was to not be held to a set a preconceived rules. Exploration about the learning experiences being developed not only took place, but was encouraged. This increased the tacit course development capabilities for all the SMEs involved in the project. So how did we apply the Agile mindset?
Instead of simply quoting back to you the Agile Manifesto, let me interpret it for the field of higher education.
- Individuals and interactions Relationships and experience over processes and tools models.
- Working software Teachable courses over comprehensive documentation comprehensive design.
- Customer Collaboration over contract negotiation status.
- Responding to change over following a plan.
Given the success of this beta experience within one program, we wanted to follow the same process across all of our programs. We presented the method to all the stakeholders, which included the program directors, senior leadership, and instructional designers. It was important that we requested their input rather than inflict the process. This helped increase engagement and willingness to try the new approaches. We adjusted the Kanban board to reflect the changes of supporting many programs. We trained the instructional designers and program directors. We then used this process for developing multiple courses.
And things started going wrong.
3.3 Problems we overcame
As beautiful as it looks on paper, and as much as we already knew it would work, implementation without stakeholders embracing and really understanding the implications of the agile mindset would not succeed. We observed teams that simply followed the process without the incremental delivery mindset, and sprints become transactional only. The curiosity of ‘why’ and collaboration that should be experienced at each sprint planning was found lacking in some of the course development teams, resulting in confused SMEs and underdeveloped courses. In fact, many courses developed using Agile tools that were purely transactional were anemic, requiring post-publication corrections of errors that caused confusion for consumers instead of investing in post-publication improvements that enhanced the learning process.
One major issue was that the kanban board wasn’t being used consistently and decisions were being made based on false data. Some instructional designers embraced the process as a model for building strong relationships, which resulted in excellent work, while others saw it as a set of templates to complete without hardly any communication, which resulted in work that almost always required fixing. Each course had its own timeline with expectations of an 8 week (4 sprint) turnaround. In the beginning, 2-3 total new course projects would start each month. However, with the growth of each program and subsequent addition of more program directors, one could expect that projection each month for each program. Due to this increase in curriculum development, an instructional designer would be contracted to represent each program. Since there were no dependencies between the program directors or between the instructional designers, tacit knowledge was being built in silos. Each program director was required to handle the curriculum of their own department with a dedicated instructional designer that simply followed the design process. Neither the program directors nor the instructional designers needed to collaborate with any other department, so challenges and solutions were discovered multiple times. Add in the constraint of all individuals being remote, some across time zones, and the princess starting growing warts.
The biggest problem for us was that not everybody truly grasped the agile mindset. For example, upon reviewing a course from the lens of a program director, I noted a cumbersome course and identified several issues that would cause re-working back through all the stages of development. When I asked the instructional designer why there was only a review of the curriculum at the end of the entire process, the mechanical and defensive response was that the process was followed. The value of the collaboration throughout development and creating a teachable course at the end of each of the four 2-week sprints was simply not understood despite the presentation of that being the purpose of the methods we were using.
For each course, the first sprint was intended to create the framework of a course that could be teachable, but would require deep expertise and time for the facilitator to teach it. The second sprint was to add methods and strategies to facilitate the course, allowing an expert to teach the course without too much effort. The third sprint was to add details and resources for the students to be extremely self-directed with their research, as well as providing enough content to make it easy for facilitators with less experience to teach the course. Finally, the fourth and last sprint was to provide the course fully equipped within a learning management system for the most effective learning support. However, the designers saw these as simply milestones where pieces of a template were completed in silos instead of facilitating the constraint identification, collaboration, and planning values found within sprints. In some cases, the writer would simply be completing a document, and then the designer would make corrections and move to the next sprint without reviewing and advising of edits. The lack of collaboration for the writer was bad enough, but to not even know of changes as the next sprint is started made it impossible for the writer to deliver effectively. The best action we found to resolve the instructional designer tendencies to prioritize the process and simply deliver an end product without collaboration and feedback from others was to mentor through behavioral modeling. Scott mentored half the designers and I mentored the other half during one of each designer’s course projects. The goal was to ensure that the instructional designers not only understood the design process, but also the purpose and agile values that drove the design process. This was our opportunity to plant the seeds of the Agile mindset, which worked to open the thinking of instructional designers and to appreciate the different ways of working outside the traditional design model of ADDIE that is so parallel to waterfall.
Another problem we initially had was when program directors were trying to discover where a course was in the flow, they could not because the card representing the course on the electronic kanban board was not being updated. The use of the cards were not valued equally among the instructional designers as they managed their own workload, especially if they felt they were personally well-organized. They simply did not understand that others needed visibility into the progress of course development. The lack of understanding that the purpose of the workflow board was for everybody to know about status led to the board becoming unreliable. Also, there were not proper stand-ups for the product owners (program directors), team leads (instructional designers), and scrummaster (program manager) who owned the virtual Kanban board. This was mostly due to the fact that all of these individuals were distributed across multiple states combined with the growing number of individuals involved in overly complex meetings. To remedy this lack of communication, the scrummaster started to meet with the product owners regularly to determine priorities and needs. This transformed what before was simply completing a form to a human connection. These developing relationships helped us recognize that communication could not be simply transactional; the context and empathy within the relationships created a sense of understanding and shared goals. This discovery led to the scrummaster then appealing to the team leads (designers) through appreciative inquiry to help her understand the status when the board would get out of sync with the project. As a consequence of these actions, two things happened: curriculum requests from the product owners became more timely and reliable, and the team leads became more active in ensuring the board represented the true status of course development. For both situations, the action was no longer a transactional task, but became a communication that helped provide reliability and visibility for all the stakeholders involved.
4. PROGRESS AND MORE GROWTH
While relationships and collaboration are essential, having consistency in our process is necessary too. As terminology vagaries within the original design stages became operationally defined and the roles / responsibilities became clearer, stakeholders found themselves able to rise above the frustrations associated with defining process (again) and able to focus on creating teachable courses with each iteration. Some teams are stronger at this than others, and there are still gaps to address, such as leadership around instructional design practices based on the organization’s culture. However, now there is the space to address these needs in a respectful and engaging manner.
As our organization continues to evolve our process and improved visualization of the process in our team Kanban board, this curriculum development structure (Figure 2) has been successfully extended to five other universities in Texas, Michigan, North Carolina, Oregon, and Kansas in the past year. The model as shown in Figure 1 has fit with relatively little adaptation at each college based on their structure. Sometimes the Dean takes on the role analogous to the product owner, others have program directors in place, and sometimes that role is represented by a committee that only wants to be involved at the beginning and end of the development. While committees can be perceived as a constraint, we have found that appropriate presentation of the material keeps the committee from meddling, while serving its role of ensuring that the course fits the university program’s vision and objectives. In all five universities, I served in both the program manager and instructional designer roles as identified in Figure 1. However, this juggling, as chaotic as it sounds, provides a sense of identity for what role or roles each contributor in the process is responsible for ensuring, which is frankly, the bottom line. Clarity of purpose within a process is crucial to effective communication, as we have found. The structure of our process allows for me, in the instructional designer role, to respect the unique aspects of how each SME(s) works. Some prefer to have working meetings. Some prefer to be shown an example and then review later. Some prefer iterative reviews within the sprint. Some like screen sharing while others prefer video sharing. This flexibility cannot be achieved without the clarity of the structure and relationship building. The universities and the SME(s) suddenly have options and preferences along with the assurance of a solid product.
It is worth noting that each of these five university relationships are less complex than our own internal organization. Surprisingly, the external university relationships are similar to the simpler context we experienced long ago as a team of three for all online curriculum at the single university in Ohio. We also find it easier coming in as consultants to those universities, as the broader application at the university in Ohio was implemented in-house, with the inherent political complications of internal implementation. When reflecting between the success of the process adoption of other universities and the internal challenges presented in this experience report, the following hypothetical scenario comes to mind. What if our administration had engaged in the adoption process and mindset once the expansion of the hierarchy had taken place for curriculum development consensus? Transitioning one small team handling all of the curriculum for over 4 years to a number of program directors responsible for their own program’s curriculum requires a plan to ensure consensus about curriculum development approaches, which was a missing component in our initial transition to take on a larger charter. This transition, starting with a single team and scaling to multiple teams across a department, led to a process and mindset developed solely as a grassroots effort and spread through influence. This is very different than our consulting relationships, where leadership assumed it is a necessary approach to adopt the process beginning with leadership. We believe that if the highest levels of administration would have had a similar eagerness and involvement for adoption as we had with the consulting universities, it is possible that stakeholders involved with the process may have more readily accepted a new way of thinking. Even more, if the mindset had been adopted and modeled across the departments for communication and smaller team processes first, there is a stronger likelihood that a cross-departmental process would have been more easily accepted as an iterative step forward in the culture rather than a major shift in both thinking and working.
5. SOME CONCLUSIONS
The challenges for implementing Agile methods in our context are getting easier as the mindset becomes more prevalent. As Agile influencers in higher education for curriculum design, Scott and I focus on the need for delivering teachable courses quickly, and to do that effectively requires collaboration to be more than just transactional relationships. The fast delivery sells the Scrum model easily. Flexibility with how that model transpires for different teams has been an important part of a successful adoption though, with the focus being on effective communication and decision-making. We intentionally modeled the process and mindset with each program director, and we mentored each contracted instructional designer. The scrummaster (program manager) was a keystone for all stakeholders, so influence there was especially critical. However, until the Agile mindset, rather than buzzwords and methods, is ubiquitous, the focus on mindset is an essential consideration for organizational change effort to be embraced. In one regard, the feeling is one of failure that the agile methods that worked so well for us did not get the reception desired for adoption across all of the academic programs. In another regard, there is a feeling of success as the instructional design model parallel to waterfall, ADDIE, is not even considered in the organization.
There’s a valuable recognition that the status quo has been shifted, but that we still must stay focused on developing the agile mindset; otherwise, change simply becomes new procedure or process without understanding why. However, making this a part of an organization’s culture appears to require three aspects that we have retrospectively observed. First, there is a need for intentional evolution as the organisation grows. Second there is the need for senior leadership to be involved throughout, especially during the time of growth. Third and final, the speed of change needs to be managed intentionally between slow evolution and the occasion burst of fast change.
6. ACKNOWLEDGEMENTS
Our thanks go to Rebecca Wifs-Brock for her patient guidance in this paper’s development as shepherd. Her expertise and interdisciplinary perspectives allowed us to share the experience in a way that benefits industries outside of higher education. Also, gratitude goes to Dr. Ingrid Buch-Wagler for providing insights as to the flow and editing. Additionally, we are very appreciative of the experiences Scott and I have had across the various universities as it has given us continual opportunity to experiment and develop curriculum in an agile mindset.
REFERENCES
Willeke, M. “Applying Agile to Instructional Design,” In L. O’Connor (Ed.) Proceedings: 2011 Agile Conference (pp. 246-251). Los Alamitos, CA: IEEE, Inc.
