In the FFG-funded research project FLOWgoesS2T, researchers and developers from XEBRIS Solutions GmbH, aiconix GmbH and RISC Software GmbH worked together to analyze telephone voice messages on Austrian traffic events. The results of these analyses should form the basis for categorizing the calls and the semi-automated creation of traffic reports, as this currently involves considerable manual effort for editors.
The aim is to simplify the interaction between editors and external sources (e.g. calls from road users) by supporting innovative AI systems. This makes it possible to quickly identify traffic reports that are particularly relevant for broadcasting, such as wrong-way driver reports, and to set up the processing process in a scalable manner. From a road user’s point of view, it is also possible to transmit a voice message in the car using intuitive voice navigation without time-consuming manual interactions that distract from the traffic situation (e.g. typing in an SMS message). Basic tools to support these tasks do exist, at least in part, but they do not take Austrian dialects into account and are therefore hardly usable in Austria.
In order to derive information from a voice call, it must first be converted from spoken to written language (text) using speech-to-text. Researchers at aiconix GmbH developed their own model for this transcription, adapted to the Austrian language and the traffic context. A particular challenge here was taking into account the many different dialects and converting them into standard German – a problem that had not yet been solved with the technologies available on the market.
In a second step, the transcribed text can be used to extract relevant information (see technical article on NLP). By combining sophisticated rule-based mechanisms and so-called transformer models (see also technical article Transformer models) – a special architecture of artificial neural networks that is particularly suitable for analyzing text data – the researchers at RISC Software GmbH developed a system for recognizing traffic-relevant text modules. For example, street numbers and names, the location, the direction of travel, but also more complex content such as the event (e.g. traffic jam, blocked lane, wrong-way driver, etc.) and the cause (e.g. rear-end collision with 2 cars, wildlife accident, construction work, etc.) can be automatically recognized by the AI system and marked accordingly.
To train such AI models, large amounts of data are required to teach the model what to do. Two public data sets with thousands of (standard) German traffic texts and associated annotations were used as a database, which were prepared for the task, standardized and adapted to the objectives of the project. Based on this, our own AI models were then trained, evaluated and finally tested on the transcripts of the calls.
When combining different systems that overlap in their areas of responsibility, not everything always runs smoothly. Thus, the combination of rule-based systems and Transformer models also posed new challenges here. Although rule-based systems offer stable and easily comprehensible results for simple cases, the structures of the rules are often too rigid to handle complex content or special cases correctly. Artificial neural networks offer far more flexibility and often surprisingly good results here, but they can also make mistakes and their behavior is more difficult to understand (see also technical article Explainable AI). By combining the two approaches, the respective advantages are to be optimally exploited in order to achieve the best possible result. The success of the prototype developed can be seen in the accuracy of around 90% (F1 score) on High German texts and around 85% on transcribed texts from calls with Austrian dialects.
As in previous collaborations, RISC Software GmbH contributed a crucial piece of the puzzle to the success of this project with its application- and solution-oriented AI expertise.
The research project was funded by the Austrian Research Promotion Agency as part of the basic program for small projects under the project number 42190322.
Senior Data Scientist
