In the FFG-funded project FLOWgoesS2T, XEBRIS, aiconix, and RISC Software GmbH analyzed Austrian traffic voice messages. The results of these analyses should form the basis for categorizing the calls and the semi-automated creation of traffic reports. This is important because the task currently involves considerable manual effort for editors.
The aim is to simplify the interaction between editors and external sources (e.g., calls from road users) by supporting innovative AI systems. In this way, traffic reports that are particularly relevant for broadcasting, such as wrong-way driver reports, can be identified more quickly. Moreover, the processing process can be set up in a scalable manner. From a road user’s point of view, it is also possible to transmit a voice message in the car using intuitive voice navigation. This avoids time-consuming manual interactions, such as typing in an SMS, that distract from the traffic situation. Basic tools to support these tasks do exist, but they do not take Austrian dialects into account and are therefore hardly usable in Austria.
In order to derive information from a voice call, it must first be converted from spoken to written language using speech-to-text. Researchers at aiconix GmbH therefore developed their own model for this transcription, adapted to the Austrian language and the traffic context. A particular challenge here was the consideration of many different dialects and their conversion into standard German. This problem had not yet been solved with the technologies available on the market.
In a second step, the transcribed text can be used to extract relevant information (see technical article on NLP). By combining sophisticated rule-based mechanisms and transformer models (see also technical article Transformer models), the researchers at RISC Software GmbH developed a system for recognizing traffic-relevant text modules. For example, street numbers and names, the location, the direction of travel, but also more complex content such as the event (e.g., traffic jam, blocked lane, wrong-way driver) and the cause (e.g., rear-end collision with two cars, wildlife accident, construction work) can be automatically recognized by the AI system and marked accordingly.
To train such AI models, large amounts of data are required. They teach the model what to do and improve performance. Two public data sets with thousands of German traffic texts and associated annotations were therefore used as a database. These were prepared for the task, standardized, and adapted to the project’s objectives. Based on this, the team trained its own AI models, evaluated them, and finally tested them on the transcripts of the calls.
When combining different systems that overlap in their areas of responsibility, not everything always runs smoothly. Thus, the combination of rule-based systems and transformer models also posed new challenges. On the one hand, rule-based systems offer stable and easily comprehensible results for simple cases. On the other hand, the structures of the rules are often too rigid to handle complex content or special cases correctly.
Artificial neural networks, however, offer far more flexibility and often surprisingly good results. Nevertheless, they can also make mistakes, and their behavior is more difficult to understand (see also technical article Explainable AI). By combining the two approaches, the respective advantages are optimally exploited to achieve the best possible result.
The success of the prototype developed can be seen in the accuracy of around 90% (F1 score) on High German texts. Moreover, the system still reaches around 85% on transcribed texts from calls with Austrian dialects.
As in previous collaborations, RISC Software GmbH contributed a crucial piece of the puzzle to the success of this project with its application- and solution-oriented AI expertise.
The research project was funded by the Austrian Research Promotion Agency as part of the basic program for small projects under the project number 42190322.
Senior Data Scientist
