STADT:up Research Project for Autonomous Driving

STADT:up is a comprehensive, multi-year collaborative project involving 22 partners from industry and research. Launched in January 2023 and funded by the Federal Ministry for Economic Affairs and Climate Action (BMWK), the project will run until December 2025. The goal of STADT:up is to design consistent, scalable solutions for future urban mobility, enabling vehicles to navigate safely through complex inner-city traffic situations. Within the project, user-centered concepts and pilot applications for automated driving in urban areas are being developed. The focus lies on interactions with vulnerable road users (pedestrians and cyclists) and complex situations, particularly in connection with AI-based methods. In this progress report, we would like to provide initial insights into our research work at STADT:up.

STADT:up is structured into five subprojects. Ergosign is specifically involved in the two subprojects: Human Factors and Automated Driving. In the first subproject, we focus on the design and prototypical development of interaction concepts that support the explanation of automation behavior. The goal is to strengthen the passengers' sense of trust and safety in complex inner-city traffic situations. In the Automated Driving subproject, the focus is on demonstrating these concepts through simulator and real-vehicle studies, which we carry out in close collaboration with our research partners. As part of this, we are developing an Engineering UI that processes sensor and vehicle information in real time, providing system engineers with valuable information and functions during the development process.

With a profound analysis of literature and related works, we lay a solid foundation for the human-centered development of interaction concepts. Alongside a sense of safety and enjoyment of use, trust in automated vehicles (AV) remains one of the central challenges regarding user acceptance. Here, the significance of the in-vehicle HMI cannot be underestimated. In autonomous vehicles, these serve as an indispensable interface between humans and technology, compensating for the absence of a human driver. To address trust and safety concerns—especially in critical situations—adequate system feedback communicated via the HMI is essential. For instance, visualizing the vehicle’s sensor-based perception in real time can provide explanations for system behavior.

Complementing the traditional literature review, we always keep an eye on the state of the art in technology and industry. We continuously analyze existing and new HMI and visualization concepts, focusing primarily on the conceptual and visual aspects of the interfaces under consideration. We examine concepts from leading AV and AI technology companies, investigating the representation of surroundings, routes, or trajectories, as well as the use of various perspectives. Furthermore, we look at the interplay between system feedback and other vehicle and route information or additional notification elements.

In the spirit of the human-centered design process, we do not just talk about (potential) users—we talk with them. Within the framework of STADT:up, we combined semi-structured interviews with co-creation methods for this purpose. Through these 'co-creation interviews,' we captured the needs and requirements of future users for the interaction concept. We translate these results into personas and user journeys and derive targeted design recommendations from them.

When developing and evaluating future concepts, we initially focus on four common inner-city traffic situations. These scenarios were selected based on the following criteria:

• High complexity of the urban traffic situation,

• Involvement of vulnerable road users (VRUs),

• Diversity of possible situational behaviors,

• Frequency of the situation in inner-city traffic.

Using these selected scenarios, the complex challenges of urban road traffic and the resulting requirements for (internal) information and communication concepts can be illustrated, analyzed, and discussed.

We have already translated the results of our requirements analysis into initial interaction concepts based on the defined scenarios. These concepts explain automation behavior in a clear and efficient manner. In other STADT:up subprojects, possibilities for identifying the intentions of other road users at an early stage are being investigated. Their findings will be integrated into our interaction concepts in the future. This approach creates modular interaction building blocks for internal vehicle communication. The modular structure allows relevant information to be targeted and delivered to vehicle occupants depending on the context, user group, and active automation level.

Initial design ideas already exist as mock-ups and wireframes and will be iteratively refined in the following steps based on the analysis results. In the next phase, we will develop interactive prototypes to be explored and evaluated in simulator and Wizard-of-Oz studies.

To test and demonstrate our concepts, we focus in particular on interactions with vulnerable road users. For efficient collaboration with our project partners, the focus is currently on developing an Engineering UI.

The Engineering UI will have the following tasks, among others:

• Preparation and visualization of sensor data from partner systems.

• Providing suitable functions for data acquisition and system demonstration.

• Configuration and demonstration of HMI modules for end users in the test vehicles.

What have we already achieved? So far, we can report the following about these ambitious goals:

1. Requirements analysis

As part of the project kick-off, Ergosign moderated a world café discussion that provided input on the requirements specification for the Engineering UI from the user's perspective. Building on this, detailed framework conditions and requirements were recorded in various joint workshops and meetings. We bundle the results into personas, journey maps, and problem statement maps. These artifacts lay an essential foundation for the successful conception, development, and integration of the (engineering) UIs.
We conducted a comprehensive technology and market analysis of available frameworks as part of the requirements analysis. The Robot Operating System (ROS), the debugging and visualization framework (Foxglove), and the tools for visualizing sensor data and researching the required hardware components were particularly exciting. We are coordinating closely with the cooperation partners such as for example the Intelligent Vehicles Lab the Munich University of Applied Sciences, who are setting up test vehicles as part of STADT:up, in which we will integrate our HMI components.

2. Focus scenarios

Ergosign's scenarios were chosen with a focus on vulnerable road users (see above). For the development and integration of the HMIs/UIs, we will also consider the scenarios of the cooperation partners, in whose test vehicles the HMI concepts will be integrated.

3. Interaction design: initial concepts and test set-up

Based on the (continuous) requirements analysis and coordination with the project partners, we carried out the first ideation workshops. We visualized the resulting ideas and concepts in scribbles and initial wireframes. These were used to coordinate with the developers and project partners in future concept workshops. In the further steps, the concepts are converted into initial prototypes, evaluated, optimized if necessary, and finally implemented. The current focus is the Engineering UI to create a lightweight and compact UI for vehicle engineers, developers, and designers. In addition, the Ergosign developer team has already dealt intensively with the framework conditions for integration into the partner test vehicles and implemented its test set-up.