Automated Detection of Fake News: How AI Fact-Checking Works

Why Trust in the News Is Dwindling—and How AI Can Help

The World Economic Forum’s Global Risks Report 2026 identifies misinformation and disinformation as one of the greatest global risks. The urgency of this issue is also evident in Austria: Social media has become the second most common primary news source for the population—and the dominant one among young adults—while not even half of those surveyed trust the news most of the time. A study by the Universities of Salzburg and Linz also shows that many Austrians are unable to reliably identify fake news. The key factor here is not so much education as digital literacy, which can be learned. This is precisely where automated fact-checking tools can help.

Step by Step to Valuation: How the Pipeline Works

The prototype operates in three consecutive steps, much like a human fact-checker—only faster. First, it searches an archive of Austrian news articles for thematically relevant sources. Then, it analyzes the stance these articles take on the claim. Finally, it weights the results based on the credibility of each source. The result: a reasoned assessment of the statement as supported, refuted, or unclear.

Find Related Articles: Search the News Archive

Searching through thousands of articles and filtering out the most relevant ones requires precise methods. To do this, the system uses proven techniques from information processing: TF-IDF and BM25 compare texts based on the frequency and distribution of keywords, similar to a search engine, but limited to a local archive. In addition, a cluster-based approach was tested, which first groups articles by topic before the actual search begins.

Support or refute? How AI identifies the stance of a text

Finding a relevant article isn’t enough. What matters is the stance it takes on the claim. Does it support it? Does it contradict it? Or does it remain neutral? A language model handles this assessment. It compares two texts and determines their logical relationship. Several models were tested, including specialized NLI (Natural Language Inference) models and well-known language models such as GPT-5. An ensemble approach (the combination of multiple models) yielded the most reliable results.

Credibility Matters: Why Not All Sources Are Equal

A key problem with many fact-checking systems is that they treat all sources equally. Yet an article from a reputable daily newspaper carries more weight than an anonymous blog post. The system therefore takes four factors into account when weighting sources: overall trustworthiness (editorial standards, transparency), historical reliability, the quality of the articles’ content, and external evaluations by independent third parties. Sources with low scores have a significantly smaller impact on the overall result.

Understanding Rather Than Just Trusting: Why AI Must Explain Its Decisions

A system that only outputs “true” or “false” is hardly convincing. Only a comprehensible explanation builds trust, which is why Explainable AI (XAI) is a central component of modern AI systems. Studies show, however, that the mere presence of an explanation has little effect on agreement or compliance and varies depending on the format and the user; usefulness, comprehensibility, accuracy, trustworthiness, and design are particularly important in this regard. Furthermore, AI explanations carry the risk of over-reliance if they are flawed. Contrastive explanations, which present arguments for and against a claim, can counteract this while ensuring that explanations are aligned with the actual workflow of fact-checkers.

Focus on Austria: Why Local Sources Are Especially Important

Most fact-checking systems were developed for English-language media. They cannot be easily adapted to Austria, either in terms of language or content. The prototype therefore relies on a specially curated archive of Austrian news sources: ranging from national daily newspapers to regional media. This local focus makes the prototype particularly relevant for Austrian users.

Evaluation: What the prototype can do and what it can’t do yet

In tests, the prototype was able to reliably find relevant articles and correctly assess the tone of the texts in many cases. It works particularly well with clearly worded, fact-based claims. It becomes more difficult with opinion-based statements or very recent events for which there are still few articles in the archive. These limitations realistically demonstrate where automated systems stand today and where human judgment is still required.