Researchers and developers from XEBRIS Solutions GmbH, aiconix GmbH and RISC Software GmbH worked closely together on the FFG-funded research project FLOWgoesS2T. Their aim was to systematically analyze telephone voice messages on Austrian traffic events. This created a basis for the categorization of calls and the semi-automated creation of traffic reports. Previously, this process involved considerable manual effort for editors.
The project also aims to simplify the interaction between editors and external sources, such as calls from road users. To this end, innovative AI systems are used to quickly identify particularly relevant messages such as wrong-way drivers. This allows the processing procedure to be scaled more effectively.
For road users, FLOWgoesS2T also makes it possible to send voice messages intuitively from the car. This eliminates the need for time-consuming or distracting manual input such as typing a text message. Although basic tools already exist, they hardly take Austrian dialects into account and can therefore only be used to a limited extent.
In order to extract information from a voice call, the system first converts it into written text using speech-to-text. Researchers at aiconix GmbH developed their own model for this purpose, which was specially adapted to the Austrian language and the traffic context. A major challenge here was the large number of dialects and their conversion into standard German – a problem that no available technology has yet been able to solve satisfactorily.
In the second step, the project team uses the transcribed text to extract relevant information(see technical article NLP). Here, RISC Software GmbH combines rule-based mechanisms with modern transformer models(see also technical article Transformer models). This special architecture of neural networks is ideal for analyzing text data. As a result, the system automatically recognizes traffic-relevant text modules such as street numbers, location information, driving directions as well as more complex content such as events (e.g. traffic jams, closures, wrong-way drivers) and causes (e.g. rear-end collisions, wildlife accidents or construction work).
For an AI model to work reliably, it needs large amounts of data. The project team therefore used two public data sets with thousands of standard German traffic texts and associated annotations. This data was processed, standardized and adapted to the project goals. The team then trained, evaluated and tested their own AI models on the transcripts of the calls. In addition, the models were specifically expanded to be able to handle Austrian dialects.
When combining different systems that overlap in their areas of responsibility, not everything always runs smoothly. The combination of rule-based systems and the Transformer models also presented new challenges here. Although rule-based systems offer stable and easily comprehensible results for simple cases, the structures of the rules are often too rigid to handle complex content or special cases correctly. Artificial neural networks offer far more flexibility and often surprisingly good results, but they can also make mistakes and their behavior is more difficult to understand(see also the technical article Explainable AI). By combining the two approaches, the respective advantages are to be optimally utilized in order to achieve the best possible result. The success of the developed prototype is shown by the accuracy of about 90% (F1 score) on High German texts, and about 85% on transcribed texts from calls with Austrian dialects.
Wie bereits in den vergangenen Kooperationen hat RISC Software GmbH auch in diesem Projekt mit seinem anwendungs- und lösungsorientierten KI-Know-how einen entscheidenden Puzzle-Stein zum Erfolg beigetragen.
The research project was funded by the Austrian Research Promotion Agency as part of the basic program for small projects under project no. 42190322 promoted.
Senior Data Scientist
