This cluster focuses on the engineering of software systems, which are large, constantly changing, adaptive, and as a consequence, systems of socio-technical systems. Through empirical studies of industry practice, challenges are identified, improvements in terms of software management, engineering and technology are designed and validated, thus generating both practical improvements and advancing the research excellence front.
Vision for the cluster
Background: With the increasing size of software systems and the emergence of increasingly autonomous systems, future software engineering practices will exhibit the following characteristics:
- All software (including safety critical software) will be continuously deployed (at least once per agile sprint) in virtually all systems while maintaining important properties like safety
- R&D teams will adopt experiment-driven development practices such as A/B testing for feature development and “minimal viable product” (MVP) approach for new product development
- Systems will use streaming analytics inspired approaches to incorporate data-driven behavior for general systems and for personalization purposes
- Systems will autonomously experiment with their own behavior to improve the value delivered to customers and other stakeholders
- Collaboration with other systems in a System of Systems setting will be exploited to enhance the capabilities of the system to sense and collect information about its environment
- Families of similar systems will learn from each other automatically
Although individual researchers and professionals in industry are researching and building elements of this vision, it becomes clear that there are no development approaches that combine human experimentation, autonomous experimentation and data-driven behavior in an integral approach including architecture and development processes. As the Swedish software‐intensive systems industry continues to shift R&D resources from mechanics and electronics to software, in response to the growing software size and the growing role of software in achieving differentiation and value delivery, we need to be world-class at building systems following the aforementioned principles at scale, with hundreds and thousands of software engineers involved.
Objective: The objectives of this cluster includes i) architectural styles and patterns for self-adaptive systems that employ automated experimentation to autonomously improve the performance of families of systems, and ii) evidence-based engineering approaches that uniformly combine human-driven experimentation and automated experimentation.
Connection to the other WASP Clusters: Software is involved in the implementation of any autonomous, AI and ML system. In that respect, the software engineering cluster is related to all other clusters. More specifically, there is a tight collaboration with the software technology cluster.
Demonstrations: The experimental methods are evaluated on real industrial systems with partners like Apptus technologies, Axis communications, and Sony Mobile. Particularly, the industrial PhD student from Systemite ensures a tight connection to industry.
The overall research goal is the generation of development approaches that combine human experimentation, autonomous experimentation and data-driven behavior in an integral approach. In order to realize this, we have recognized four research challenges that we study in this cluster.
Software archtectures: A self-reflective architecture is able to measure and track its own performance whereas self-adaptive refers to the ability to adjust its own behavior. To accomplish this, as well as to support experimentation, data collection needs to be an integral part of the architecture. In addition, the architecture needs to support continuous deployment and facilitate automated experimentation.
Continuous experimentation: Novel software engineering methods are required that are based on experiment-driven, rather than requirements driven development. These development methods need to support data-driven or evidence-based development, meaning the use of data from systems in the field and the behavior of users of these systems for decision making. In addition, these methods need to support the transition from human experimentation by R&D teams to automated experimentation.
Data analysts: To support the aforementioned methods for software engineering, new techniques for data analytics and use are required. These techniques are required to establish the success of autonomous behavior and determine value of new features. In addition, these techniques need to support automated experimentation.
System families: Smart systems will exist in families of dozens, hundreds or thousands of similar instances of each system. As these systems are experimenting with their own behavior, we need to develop techniques that allow the R&D teams responsible for these systems as well as the systems themselves to learn from the family of systems. The learnings need to be merged and redistributed to all systems in order to ensure continuous improvement of all systems.
The industrial challenge is predominantly one of scaling, both the engineering endeavor and the data collection and analysis practice. Already today, Swedish companies can build small autonomous systems using small teams of highly skilled engineers and researchers. These systems and teams however are not employing experimentation and data-driven techniques, but rather employ trial-and-error as their primary mechanism.
However, future industrial engineering of large software‐intensive systems, including autonomous systems, will require the involvement of hundreds and potentially thousands of engineers, and data collection from hundreds of thousands or millions of instances of the systems. Currently, there are no engineering approaches that support the future data-driven development practices outlined above and that scale to the level that will be required for large-scale industrial deployment.
This cluster is concerned with scaling, i.e. developing architectures and development practices that support large-scale development employing the trends and technologies that will become available in the coming decade.
Data‐driven continuous evolution of smart systems
David Issa Mattos (PhD student, Chalmers), Jan Bosch (advisor, Chalmers University of Technology), Helena Holmström Olsson (co-advisor, Malmö University)
The subproject is concerned with self-adaptive architectures that inherently support automated experimentation and data-driven behavior. In the first phase, the project seeks to develop new architecture styles, patterns and solutions for self-adaptive architectures and systems that address the requirements including integral data collection and automated experimentation. In the second phase, we expand the project to focus on the development methods required to develop systems using self-adaptive architecture. During both phases, industrial validation is applied to ensure industrial relevance and scalability, which particularly include collaboration with Sony and Ericsson.
Data collection and analysis for continuous experimentation
Rasmus Ros (PhD student, Lund University), Per Runeson (advisor, Lund University), Elizabeth Bjarnason (co-advisor, Lund University)
This subproject aims at exploring and advancing methods for operational data collection and analysis in the above described context of continuous deployment and experimentation. In current software engineering practices there are significant gaps between the users and the engineers, leading to suboptimal solutions. In order to bridge this gap, methods for continuous monitoring of system usage and experimentation with new and modified features has evolved. Current approaches are limited to reactive evaluation of minor variations of the software services. We aim to go beyond basic quantitative measurements of response, and explore options of qualitative analysis of user behavior, to allow proactive recommendations to be presented to the software engineers. In a first step, we survey in detail existing proposals, secondly we focus on the data collection and analysis from the architectures, mentioned above, and thirdly, integrate the recommendation system into the organic development environment.
Organic evolution of development organizations
Rebekka Wohlrab (industrial PhD student, Systemite AB, Chalmers), Patrizio Pellicione (advisor, Gothenburg University), Riccardo Scandariato (co-advisor, Gothenburg University), Ali Shahrokni (Systemite AB)
The ambition of this subproject is to conceive intelligent, integrated, and system-level development data management methods and tools to support organizations on their mission towards developing smart systems. The new advanced, distributed mechatronics systems manifesting as self-driving cars or Internet-of-things have pushed people and organizations to a different level of thinking; a level where understanding and relating to the context and the bigger picture, a system architecture, is at least as important as providing high quality in the details. This small shift in perspective demands a new way of organizing and even new methods of handling the changes in new organizations.
Data management methods and tools support development organizations in their “organic evolution” that involves and keep synchronized during evolution three distinct but interdependent structures: (i) the architecture of the system under design, development, or refinement, (ii) the structure of the organization, including also partners, subcontractors, and others, and (iii) the production infrastructure used to develop and deploy the system. This synchronization will reduce problems caused by both technical debt, i.e. when the architecture is lagging, and social debt, i.e. when the structure of the organization is missing.
Continuous Experimentation for Software Developers
Aleksander Fabijan (Affiliated PhD student Malmö University), Helena Holmström Olsson (advisor, Malmö University), Jan Bosch (Co-advisor, Chalmers University of Technology)
This subproject aims at exploring and advancing continuous experimentation techniques for software developers and other practitioners without extensive data science trainings. Experimentation has been well-adopted in software companies developing online products and other context where data science experts are present. In order to expand the experimentation capabilities and enable software developers to conduct experiments at large scale, we need to (1) understand how to set-up trustworthy experiments, (2) how to build platforms and tools that will support their execution, and (3) identifying what the benefits of running experiments at scale. In addition, this subproject aims to (4) investigate the maturity of experimentation in the software industry, and (5) the rate of advancement towards mature, experimentation-first companies. The research work in this project is done with collaboration with the following companies: Ericsson, Jeppesen, Axis, Saab, Volvo, Siemens, Bosch, Microsoft, Booking.com, Skyscanner, Netflix, and Intuit.