vizARd – Visual Augmented Reality Assistant for Spatial Mapping

Before certain operations, three-dimensional cross-sectional images, such as computed tomography (CT) or magnetic resonance imaging (MRI), are produced in order to plan the medical procedure precisely. In future, surgeons are to be supported by an augmented reality solution developed in the VizARd project. This software makes it possible to visualize this image data in an intuitive and user-friendly way directly on the patient’s skin at the site of the incision.

Soft tissue and position detection as challengesn

Although there is a lot of valid data on humans and their “inner life”, humans are anything but a rigid “object”, as the position of the skin surface, soft tissue and organs changes depending on the position of the body. The research work is based on methods from the fields of augmented reality (AR) and deep learning, which have so far been developed specifically for the field of assistance systems. What was previously used to digitize people, their tools and workpieces in the assembly hall is now being transferred to people in the operating theatre.

Another key challenge is the deformation of soft tissue, which must be taken into account when visualizing image data. The Medical Informatics research department at RISC Software GmbH has already gained valuable experience in biomechanical modeling through previous projects. The visualization of medical images directly on 3D patients – instead of on a 2D screen – forms the basis for pioneering research projects and for the improvement of existing applications in the medical field.

Unique support in preoperative planning

Based on AR technology, VizARd bridges the gap between the output of a digital workflow and a real object by combining static and mobile AR devices in a unique way. The environment is perceived by 3D sensors in real time. Novel, non-rigid registration methods allow for high-precision mapping on deformable surfaces. The exact determination of the position of devices and objects enables high-quality AR presentations. Overall, VizARd technology facilitates preoperative marking based on data from CT/MRI devices in hospitals.

Cooperation with MedUni Graz

The medical project partners from the Division of Plastic, Aesthetic and Reconstructive Surgery and the Department of Radiology at MedUni Graz contributed their expertise, in particular by helping with the practical implementation and providing the medical application scenarios.

User-friendliness as an important goal

To ensure a high level of user acceptance of the VizARd technology, surgeons were involved in the development from the very beginning. The digital agency Netural GmbH was primarily responsible for the design of the software’s user interface. The company has been conducting intensive research in the fields of e-health and medical technology for many years.

Augmented Reality interaction with vizARd data by a surgeon.

A preparation app guides the medical expert through the process of 3D MR data processing
and 3D surface acquisition of the actual patient.

After deformation to the current patient shape, abdominal vessels from MR are visualized to
support presurgical marking.

A rich AR interface allows the medical expert to overlay different data representations.

Project partners

Contact

Project management