Skip to content

Research Project nARvibrain: Improved brain tumour diagnosis and treatment

The nARvibrain project (“Augmented Reality supported Functional Brain Mapping for Navigated Surgery Preparation and Education”) aims to improve brain tumor diagnosis and treatment, increase patients’ awareness and understanding of the disease, and enhance the quality of medical education by combining modern Artificial Intelligence (AI) and eXtended Reality (XR) methods.

A brain tumor causes a considerable physical and psychological burden for the patient and entails a wide range of diagnostic and therapeutic measures. Prior to possible tumor resection, information from multimodal imaging and functional testing is routinely obtained. Both the location of the tumor and the surrounding eloquent brain regions must be considered to achieve an optimal treatment outcome. The goal is to remove as much tumor tissue as possible while preserving essential neurological functions.

The cornerstone of the nARvibrain project is the development of a holistic digital representation of a patient (“Digital Patient Model”). This model represents all relevant structural and functional image data, functional test results and predictive simulation results. A semi-automated, AI-assisted image processing pipeline is being developed that converts medical image data into tailored input for Augmented Reality rendering, simulation, and prediction packages.

nARvibrain
(c) FH JOANNEUM GmbH & Medizinische Universität Graz
nARvibrain
(c) FH JOANNEUM GmbH & Medizinische Universität Graz

(c) FH JOANNEUM GmbH & Medizinische Universität Graz

Physicians in their daily clinical practice as well as patients and medical students will benefit from the nARvibrain research project.

Improved brain tumor diagnosis and treatment

A key development of nARvibrain is the Augmented Reality guidance in functional testing, i.e., transcranial magnetic stimulation (TMS) or direct cortical stimulation (DCS). An advanced neuronavigation system (“CORTEXPLORER” of the project partner cortEXplore GmbH, www.cortexplore.com/software/) will be extended to allow precise tracking of stimulation instruments as well as quantitative measurement of behavioral responses to TMS and DCS stimulations. Overall, this will allow pre-surgical testing to be performed faster and more precisely, thereby improving treatment outcomes and patient safety.

Improved patient counseling

In nARvibrain, a prototype solution for an interactive 3D visualization of structural and functional image data is being developed. This visualization tool will help physicians to comprehensively explain complex information about the disease and the treatment plan to patients to improve their understanding of the disease and the surgical strategy.

Improved medical education

nARvibrain also contributes to increasing the quality of medical education by developing innovative teaching materials for neuroanatomy. Photogrammetric models of human brains will be virtually augmented with tumor and tractography image data. This provides the basis for a gamification-based didactic platform designed to reduce the common phenomenon of “neurophobia” among students.

The nARvibrain research project focuses on brain tumors (gliomas). The project partners are convinced that this project, with its unique combination of XR-systems and predictive guidance for medical interventions, will serve as a basis for similar applications in other clinical specializations.

This project is funded by the FFG. www.ffg.at. “ICT of the Future” programme – an initiative of the Federal Ministry for Climate Action, Environment, Energy, Mobility, Innovation and Technology (BMK).

Project partners

Logo RISC
Logo Cortex
Logo FH Joanneum
Logo MEDUNI
Logo FFG
Logo BMK

Details about the project

  • Project short title: nARvibrain
  • Project full title: Augmented Reality supported Functional Brain Mapping for Navigated Surgery Preparation and Education
  • RTI call: ICT of the Future, 10th Call 2021 (FFG)
  • Project partners:
    • RISC Software GmbH (consortium leader)
    • cortEXplore GmbH
    • FH JOANNEUM Gesellschaft mbH
    • Medical University of Graz
  • Budget volume (total): 1.6 million euros
  • of which funding (total): 1.3 million euros
  • Duration: 36 months (01.11.2022 – 31.10.2025)

Contact









    Project management

    DI Dr. Stefan Thumfart

    Project Manager & Senior Researcher