An evolutionary graph of local, national and international news events.
Researchersand developersfrom RISC Software GmbH, SCCH GmbH and Newsadoo GmbH worked closely together on the FFG-funded research project NEEED (“News-Extracted Evolving European Datasphere”). The aim was to further develop the Newsadoo platform. Newsadoo collects, analyzes and sorts news from local, national and international sources fully automatically. This enables users to receive personalized, topic-specific and decentralized news.
The partners had already improved the automatic processing of news articles and the recommendation algorithm in the TIDE pre-project. With NEEED, the Newsadoo technology reached the next level. Now the collected data can be structured and used in the form of a tag graph (“News Datasphere”). On a daily basis, information from news articles is merged into a dynamic network of related keywords. In addition, their development over time becomes visible. This allows both long-term correlations (e.g. “Rome” and “Vatican”) and short-term trends (e.g. “ChatGPT” and “Natural Language Processing” or “Queen Elizabeth II” and “funeral”) to be derived and analyzed.
Over 30,000 news articles from German and English sources are added every day. This means that the Data Sphere is constantly growing with the latest topics. With several million articles and over a million keywords, traditional methods quickly reach their limits. This is why the project relies on big data technologies. These not only enable current data to be analyzed, but also ensure scalability for the future.
The researchers used various approaches from artificial intelligence (AI), statistics and association analysis to calculate the relationships between keywords. This combination resulted in a relationship network that is stored in a graph database. A special query language provides the connection between two keywords in just a few seconds and also shows the most relevant neighbors. This creates a versatile basis that can be used for various applications in the long term – for example, for defining subject areas or discovering new tags.
generated with DALL-E
Image: Simplified representation of the News Datasphere – a dynamic network of news tags, (C) Newsadoo
Image: Simplified representation of the News Datasphere for “Vienna” – a dynamic network of news tags, (C) Newsadoo
Die neu entwickelte News Datasphere bildet die Basis für vielfältige Nachrichtenanalysen, ermöglicht das Explorieren von Themengebieten sowie die spielerische Visualisierung von Zusammenhängen im täglich produzierten News Content. Die Zusammenarbeit mit den Forscher*innen der RISC Software GmbH war, wie bereits im Vorgängerprojekt, sehr bereichernd für uns.
This project was funded by the Austrian Research Promotion Agency (FFG).
Data Scientist
