Research Paper Abstracts

JUNE 8-12, 2020

ONLINE! Copenhagen, Denmark

Research Paper Abstracts

Improving Risk Management in a Scaled Agile Environment

Eva-Maria Schön (HAW Hamburg), Dirk Radtke (OTTO GmbH & Co KG) and Christian Jordan (OTTO GmbH & Co KG).


Agile methods are designed for handling uncertainty as well as reducing risks in product development through transparency, inspection, and adaptation. Applying an effective risk management is in the nature of agile methods. However, when multiple agile teams work on the same product, a higher coordination effort is required and more formal practices are applied. The objective of this paper is to study how risk management can be improved in a scaled agile environment. Therefore, we conducted a case study in a large-sized ecommerce company and interviewed several project managers. The results show that there are differences for risk management in terms of two contexts. On the one hand, informal risk management is rated as good enough for one autonomous team. On the other hand, more formal approaches are needed, when several teams work on the same requirement. Furthermore, a tool for the support of risk management in a scaled agile environment is presented. We can conclude that hybrid development approaches consisting of agile practices and traditional practices, are beneficial, when several teams work in parallel.

Group dynamics challenges for agile leaders: The importance of team design

Lucas Gren (Chalmers University of Technology and The University of Gothenburg) and Magdalena Lindman (Volvo Cars).


When large industrial organizations change to (or start with) an agile approach to operations, managers and some employees are supposed to be “agile leaders” often without being given a clear definition of what that comprises when building agile teams. An inductive thematic analysis was used to investigate what challenges 15 practitioners working with an agile approach define in relation to group dynamics aspects. Team maturity, Team design, and Old and the new work processes were all categories of challenges related to group dynamics that the practitioners face and manage in their work-life that are not explicitly mentioned in the more process-focused agile transformation frameworks. The results suggest that leader mitigation of these three aspects of group dynamics is essential to the success of an agile transformation.

Combining User-Centered Design and Lean Startup with Agile Software Development: A Case Study of Two Agile Teams

Ingrid Signoretti (PUCRS), Larissa Salerno (PUCRS), Sabrina Marczak (PUCRS) and Ricardo Bastos (PUCRS).


The combined use of User-Centered Design and Lean Startup with Agile Development has been pointed out by the literature as a manner to boost software development. User-Centered Design principles focus on providing tools for developers to better explore the user needs and seek for a fitter solution. Lean Startup, on the other hand, supplements the triad combination by bringing the Build-Measure-Learn cycle and the concept of pivoting, either the problem understanding of the proposed solution. This paper reports on a case study of two teams of a multinational company that have been undergoing such transformation. We investigated them for six months, from the moment that team members were trained on the job to grasp the essence of using the integrated approach inspired on Pivotal Labs proposal to the time they were considered mature enough to share their experiences worth others within the organization. Through our in-depth study, we illustrate how this adoption promotes mindset, activities, practices, and techniques changes. We also report on the ‘team rhythm’ as experienced by the two teams. The paper contributes to current knowledge on the topic reporting on the settings of such transformation in a multinational company.

“I don’t understand!”: Toward a Model to Evaluate the Role of User Story Quality

Daniel Hallmann (University of Bamberg).


User stories are popular for conveying requirements in agile software projects. Despite existing quality criteria, authors make formal mistakes that result in “bad” user story quality. If developers have insufficient experience in balancing quality problems, the creation of a shared mental model is impossible, thus increasing the risk of a negative project impact. This article provides a work-in-progress research model to set these variables in relation and establish a systematic method to find answers regarding their correlation. Details on the effects support research in agile requirements engineering to gain a better understanding of cognitive processes in the comprehension of user stories. Insights can also help to build design recommendations and AI tools for improving user stories. A first evaluation of the model provides promising insights into the behavior and a basis for future research.

Hypotheses Elicitation in Early-Stage Software Startups Based on Cognitive Mapping

Jorge Melegati (Free University of Bozen-Bolzano) and Xiaofeng Wang (Free University of Bozen-Bolzano).


Software startups develop innovative products for which there are no customers to refer to in order to collect requirements. Thus, usually these companies develop a set of features without better understanding customer needs in order to make their products available. An experiment-based approach where experiments are created to validate the team’s hypotheses about the customer and market would increase the startup’s chance of success or, at least, the speed on which the team realizes there is no need for the product, leading to an economy of resources. The first step along this direction is to elicit the hypotheses on which experiments will be built. Software startups base their products on business assumptions, but there is a lack of understanding how these assumptions are formed and how hypotheses could be systematically elicited from them. In order to achieve this objective, we performed an empirical study in two phases where each phase consisted of a multiple-case study. In the first phase, we explored based on which assumptions startups define their product. The outcome indicated that these companies developed their products based on founders’ assumptions derived from their previous experience. Based on that, in the second phase, we investigated the potential of cognitive mapping to systematically elicit hypotheses from founders’ assumptions. Our results indicate that cognitive mapping can serve as the basis of a tool to support early-stage software startups to elicit hypotheses.

Agile software development practices and success in outsourced projects: The moderating role of requirements risk

Oliver Krancher (IT University of Copenhagen).


Although agile practices are gaining popularity, relatively little is known about how particular agile practices affect the success of outsourced software projects and how these effects depend on requirements risk. Data from a matched-pair survey on 60 information systems projects shows positive effects of continuous integration and joint decision making and negative effects of continuous analysis on project success. Moreover, interaction analyses show that the positive effects are enhanced and negative effects dampened when requirements risk is high. These findings suggest that outsourced projects may best combine detailed up-front analysis with joint decision making and continuous integration. The findings also provide support for the largely untested assertion that agile practices help cope with changing requirements.

“When in Rome, Do as the Romans Do”: Cultural Barriers to Being Agile in Distributed Teams

Darja Smite (Blekinge Institute of Technology), Javier Gonzalez-Huerta (Blekinge Institute of Technology) and Nils Brede Moe (SINTEF ICT).


With the growing interest of adopting agile methods in offshored process, many companies realized that the use of agile methods and practices in companies located outside Europe and North America, the location of early adopters of agile methods, may be challenging. Asian region, the main des-tination of offshoring contracts, have received particular attention, due to the big cultural differences. Critical analysis of related studies suggests that impeding behaviors are mostly rooted in the hierarchical culture of Asian organizations and related management behavior of command-and-control. But what happens in distributed projects with a more empowering onshore management? In this paper, we present the findings from a multiple-case study of mixed DevOps teams with members from a mature agile company located in Sweden and a more hierarchical offshore vendor from India. Based on two focus groups we list culturally distinct behaviors of offshore engineers that were reported to impede agile ways of working. Furthermore, we report the findings from surveying 36 offshore team members from five mixed DevOps teams regarding their likely behavior in situations reported to be problematic. Our findings confirm a number of previously reported behaviors rooted in cultural differences that impede the adoption of agile ways of working when collaborating with offshore engineers from Asia. At the same time, our survey results suggest that among the five surveyed teams there were teams that succeeded with the cultural integration of the offshore team members. Finally, our findings demonstrate the importance of cultural training especially when onboarding new team members.

A Quantitative Exploration of the 9-Factor Theory: Distribution of Leadership Roles between Scrum Master and Agile Team

Simone V. Spiegler (University of Stuttgart, Robert Bosch Automotive Steering GmbH), Daniel Graziotin (University of Stuttgart), Christoph Heinecke (Robert Bosch GmbH) and Stefan Wagner (University of Stuttgart).


A number of qualitative studies find that team leadership is one essential success factor for evolving into a mature agile team. One such  qualitative  study  suggests  the  9-Factor  Theory  of  Scrum  Master roles, which claims that the Scrum Master performs a set of 9 leadership roles  which  are  transferred  to  the  team  over  time  [14].  We  aimed  at conducting  a  quantitative  exploration  that  examines  the  presence  and change of the 9-Factory Theory in relation to team maturity. We conducted an online survey with 67 individuals at the conglomerate Robert Bosch GmbH. Descriptive statistics reveal that the Scrum Master and the agile team score differently on the 9 factors and that the Scrum Master  role  is  most  often  distributed  in  teams  that  had  been  working between 3 and 5 months in an agile manner. Yet, we also find that the leadership roles predominantly remain with one dedicated Scrum Master. Based on our results we suggest to group the 9-Factor Theory into three clusters: the Scrum Master is rather linked to psychological team factors (1), while the team tends to be linked to rather product-related factors (2). Organizational factors (3) are less often present. Our  practical  implications  suggest  an  extension  of  the  Scrum  Master description. Furthermore, our study lays groundwork for future quantitative testing of leadership in agile teams.

An Agile Implementation and Expansive Learning: Identifying Contradictions and their Resolution using an Activity Theory Perspective

Pritam Chita (Edinburgh Napier University), Peter Cruickshank (Edinburgh Napier University), Colin Smith (Edinburgh Napier University) and Kendall Richards (Edinburgh Napier University).


A key challenge organisations face when transitioning to agile delivery methods is that of quickly and effectively learning new ways of working. This study posits that fundamental historical, cultural and behavioural aspects affect the transition and contribute to the poor performance of many agile implementations. In order to address such factors, this study applies a modified Activity Theory (AT) based framework to a case study agile implementation within a large public sector organisation. An activity is closely defined and then six generic activities associated with all agile implementations are identified. These are validated against the agile maturity model literature and a set of evaluation criteria of contradictions, congruences and collaborative activity is established. Evidence is gathered from observation and participant interviews and the analytical framework is used to surface learning & development obstacles and issues within an expansive learning cycle. The study argues that analysis via this modified implementation of AT brings original insight. Initial findings indicate that there are relatively few problematic issues associated with the use of agile tools and techniques themselves and that most problems arise at the interface where the “changed” (more agile) delivery teams meet the organisation’s behavioural norms and practices.

Onboarding: How Newcomers Integrate into an Agile Project Team

Peggy Gregory (University of Central Lancashire), Diane Strode (Whitireia Polytechnic), Raid Alqaisi (Independent), Helen Sharp (The Open University) and Leonor Barroca (The Open University).


Although a stable team is deemed optimal for agile project success, new team members need to join ongoing agile projects. Newcomers must rapidly assimilate into the organisational and project environment while learning how to contribute effectively to the project and integrate into the team without seriously interrupting project progress. This paper addresses how newcomers integrate into an established agile project team and the challenges newcomers and the team face during this process. This paper is a single case study of a co-located agile project team in a large IT department who regularly onboard inexperienced newcomers. We found a mixture of traditional onboarding practices and specific agile practices contribute to the onboarding process. Onboarding challenges include empowerment and mindset change, accommodating part-timers, conveying agile principles, and adjusting to changes in team composition.

“This is just a prototype”: How Ethics Are Ignored in Software Startup-like Environments

Ville Vakkuri (University of Jyväskylä), Kai-Kristian Kemell (University of Jyväskylä), Marianna Jantunen (University of Jyväskylä) and Pekka Abrahamsson (University of Jyväskylä).


Artificial Intelligence (AI) solutions are becoming increasingly common in software development endeavors, and consequently exert a growing societal influence as well. Due to their unique nature, AI based systems influence a wide range of stakeholders with or without their consent, and thus the development of these systems necessitates a higher degree of ethical considera-tion than is currently carried out in most cases. Various practical examples of AI failures have also highlighted this need. However, there is only limited research on methods and tools for implementing AI ethics in software development, and we currently have little knowledge of the state of practice. In this study, we explore the state of the art in startup-like environments where majority of the AI software today gets developed. Based on a multiple case study, we discuss the current state of practice and highlight issues. The cases underline the complete ignorance of ethical consideration in AI endeavors. We also outline existing good practices that can already support the implementation of AI ethics, such as documentation and error handling.

On the Use of Design Thinking: A Survey of the Brazilian Agile Software Development Community

Matheus Prestes (Pontifical Catholic University of Rio Grande do Sul (PUCRS)), Rafael Parizi (Pontifical Catholic University of Rio Grande do Sul (PUCRS)), Sabrina Marczak (PUCRS) and Tayana Conte (Federal University of Amazonia).


Design Thinking (DT) has been chosen as an approach to problem-solving by many software development professionals, as the literature on the subject has shown. However, there are divergences concerning the techniques performed, the steps followed, and the way to implement this approach, as it proposes itself, to be divergent to generate numerous alternatives and, also, convergent, to find a solution. For this reason, aiming to know the different ways that the software companies have been implemented DT, this paper presents a survey answered by 127 professionals from the Brazilian software industry. The results report a variety of scenarios in which DT has been applied: more than ten different models (sets of steps) are followed by the professionals; more than 50 techniques have been used, mainly, for meeting the needs in the process, according to the context of use and based on previous experiences. We also present 29 computational tools that, according to the respondents, assist the execution of DT, in addition to the integration with agile methods, allowing them to generate ideas and solutions, to explore and understand the problem.

Characterising the Quality of Behaviour-Driven Development Specifications

Leonard Peter Binamungu (Department of Computer Science, University of Manchester), Suzanne M. Embury (Department of Computer Science, University of Manchester) and Nikolaos Konstantinou (Department of Computer Science, University of Manchester).


Behaviour-Driven Development (BDD) is an agile testing technique that enables software requirements to be specified as example interactions with the system, using structured natural language.  While (in theory) being readable by non-technical stakeholders, the examples can also be executed against the code base to identify behaviours that are not yet correctly implemented.  Writing good BDD suites, however, is challenging.  A typical suite can contain hundreds of individual scenarios, that must correctly specify the system as a whole as well as individually.  Despite much discussion amongst practitioners and in the blogosphere, as yet no formal definition of what makes for a high quality BSS suite has been given.  To shed light on this, we surveyed BDD practitioners, asking for their opinions on the quality criteria that are important for BDD suites.  We proposed, and asked for opinions on, four quality principles, and gave practitioners the option to add more principles of their own.  This paper reports on the results of the survey, and presents an approach to defining BDD suite quality.

Α Survey on System Testing Methodologies

Rafaella Sophocleous (University of Cyprus) and Georgia Kapitsaki (University of Cyprus).


Testing is an important phase of every software system, as it can reveal defects early and contribute to achieving high software quality. In Agile development, testing is continuous in every development iteration, and various techniques are used to minimize the errors that will enter the production environment. Organizations are usually relying on a specific testing technique in this process of quality assurance. However, a combination of techniques may prove more beneficial to the organization, as it might give them the chance to discover a larger number of defects early. In order to examine the above, in the current work we present a survey on the use of system testing methodologies with most participants using Agile development processes. We have gathered data from 252 individuals. Our data reveal current trends in testing, such as whether requirements are used in the test case definition and whether the testing techniques used are affected by other parameters, whereas we focus on examining the combination of using smoke testing and regression testing. We also demonstrate an industrial use case, showing that the combination of some and regression testing improve the testing process by reducing the number of defects identified by the customer.

Results from a Replicated Experiment on the Affective Reactions of Novice Developers when Applying Test-Driven Development

Simone Romano (University of Bari), Giuseppe Scanniello (University of Basilicata), Maria Teresa Baldassarre (University of Bari), Davide Fucci (SERL Sweden, Blekinge Institute of Technology) and Danilo Caivano (University of Bari).


We present the results of a replicated experiment that studies the affective reactions of novice developers when applying TDD (Test-Driven Development) to develop software. Developers in the treatment group carried out a development task using TDD, while those in the control group used a non-TDD approach. To measure the affective reactions of developers, we used the Self-Assessment Manikin instrument complemented with a liking dimension. The most important differences between the baseline and replicated experiments are: (i) the kind of novice developers involved in the experiments—third-year vs. second-year undergraduates in Computer Science from two different universities; and (ii) their number—29 vs. 59. The results of the replicated experiment do not show any difference in the affective reactions of novice developers. Instead, the results of the baseline experiment suggest that developers seem to like TDD less as compared to a non-TDD approach and that developers following TDD seem to like implementing code less than the other developers, while testing code seems to make them less happy. Summarizing, these findings of the baseline experiment are not confirmed by the replicated one.

Large-Scale Agile Transformation: A case study of transforming business, development and operations

Nils Brede Moe (SINTEF) and Marius Mikalsen (SINTEF ICT).


Today, product development organizations are adopting agile methods in units outside the software development unit, such as in sales, market, legal, operations, and in work with the customer. This broader adoption of agile methods has been labelled agile transformation and is considered a particu-lar type of organizational change. So far there is little research-based advice on conducting such transformations. Aiming to contribute towards provid-ing relevant research advice on agile transformation, we apply a research-based framework for evaluating organizational agility on a product devel-opment program in a maritime service provider. We found that doing an ag-ile transformation involves a number of significant challenges, such as hav-ing a shared understanding of the problem, getting access to users and get-ting commitment to change that needs done. In order to overcome such chal-lenges, we discuss the need for a holistic and integrated approach to agile transformation involving all the units linked to software development.