Artanim XRay

Artanim XRay

by | Jun 25, 2026

Artanim XRay

Revealing the data underneath

Project Info

Start date:
June 2026

End date:
–

Funding:
–

Coordinator:
Artanim

Back

Summary

Besides its use in the entertainment industry, motion capture has proven itself to be an invaluable tool in other domains such as sports science, robotics and ergonomics, as well as for various healthcare applications.  

Particularly in healthcare scenarios however, one significant hurdle remains; equipping a patient with markers is slow and cumbersome. As such, even when a marker-based motion capture system is available, physicians for example often prefer to rely on a goniometer for manual joint angle measurements.  

This is a typical scenario where real-time markerless motion capture would be of great benefit. Tracking a patient requires nothing more than a literal click of a button, and in most cases can simply happen in the outfit they showed up in.  

But what if besides showing the physician live data on a monitor, we could also bring the tracked data to life in the examination room?  

In this project we’re developing Artanim XRay, an Augmented Reality (AR) application that serves as a testbed to evaluate how real-time markerless motion capture data can be made more insightful in healthcare contexts. Similar in spirit to our earlier HoloMed project, but benefitting from a decade of technological advancement, Artanim XRay combines the “passthrough” functionality of modern XR headsets with the real-time markerless motion capture of one or more subjects and overlays the tracked skeleton on top of the live AR view of the subject.  

The application allows for the spatial selection of relevant joints – for example those for which a patient could be undergoing physical rehabilitation – and offers a set of tools to visualize relevant parameters such as joint angles and joint paths over time in real-time.  

We’re looking to expand the toolset throughout the project and aim to establish how this combination of technologies can be beneficial for healthcare, with an eye on education, evaluation and rehabilitation.  

Curate XR

Curate XR

by | Oct 29, 2025

Curate XR

XR experience guidance using real-time LLM-driven conversational agents

Project Info

Start date:
January 2025

End date:
December 2026

Funding:
–

Coordinator:
Artanim

Back

Summary

Imagine yourself inside a museum environment, whether real or entirely virtual. The elegant space is filled with fascinating works of art, some which you immediately recognize, but who was the artist again? And others grab your attention but are entirely unfamiliar. If only someone was around to tell you about the artists behind the works of art or provide you with details on the works themselves. Thankfully a virtual museum guide is in the room with you.

One of the drawbacks of more traditional approaches to such a museum guide scenario is the need to establish all the possible questions and responses up front, quickly leading to intractable production constraints, and an end-result that while informative may feel unnatural or robotic given that it’s entirely predetermined. The goal of our CurateXR project is to study how a modern-day generative AI chatbot backed by a Large Language Model (LLM) can be used to provide a real-time and entirely natural means of interacting with a virtual agent in such virtual (VR) or augmented reality (AR) scenarios. Powered by an animation system making use of Motion Matching and pathfinding to naturally navigate the space, the guide will happily stroll along with you.

All interactions happen through natural speech. Just ask any question you may have, and the guide will reply without delay, taking into consideration the context of the space it has been provided with up front, as well as real-time contextual clues provided to it such as your location and the artwork you’re looking at. Not only will the guide engage in a natural conversation with you, but if you feel more comfortable speaking in another language, just ask him to do so. The use of a generative AI chatbot resolves many issues around the barriers to entry users may feel when interacting with extended reality applications for the first time, and it provides a level of accessibility which is hard to replicate otherwise.

Generative AI chatbots aren’t just limited to conversations. They have the ability to turn your words into concrete actions. To demonstrate this, at one end of the museum space you will find a virtual sculpture experience, controlled by another agent. Through natural speech you are not just able to summon three-dimensional shapes in your desired color, but you can have them move as you desire. And if you’re not satisfied with the results you achieved, there is no need to start from scratch. Just inform your virtual helper about what changes you would like to make, and you’ll see the relevant updates happen before your eyes.

Integrating a generative AI chatbot into your VR scenarios can level up your experience by providing an entirely natural means of engagement. Just put your headset on and go. And while the museum scenario was chosen as the use-case for this project, the potential range of scenarios that can be supported – from training and education to rehabilitation, entertainment and many more – is virtually endless.

MoRehab XR

MoRehab XR

by | Oct 3, 2021

MoRehab XR

Motion tracked rehabilitation through extended realities

Project Info

Start date:
August 2020

End date:
December 2026

Funding:
–

Coordinator:
Artanim

Back

Summary

After a stabilization surgery or a traumatic accident, physical rehabilitation is often required to restore the joint’s normal function. This process requires very regular and repetitive training sessions. The regularity and involvement of the patient being critical for success of the procedure, maintaining them interested and motivated while repeating similar exercises over and over is one of the main challenges of the physical therapist. During these sessions, it is also difficult to objectively monitor small progresses or the lack of the latter. On the other end of the spectrum, some video games have proved to be very effective in motivating people to perform pretty repetitive tasks on their controllers in order to travel through a story or a challenge.

The goal of this project is to use the captivating capabilities of video games to entertain and motivate the patients through their rehabilitation, by designing a set of “exercise games” (called exergames) that will be won by performing the correct routines required for a proper physical rehabilitation. Through the use of modern motion tracking technology, the exergames will also be able to track the evolution of the physical recovery of the patients on a session per session basis, to adapt the challenges that the patients will face based on their needs and actual capacities, and if need be to alert the medical personnel early if some problems persist, or if no progresses are observed.

In this project, Artanim is in charge of building a platform suitable for these exergames, as well as designing and developing these exergames, and implementing the tools for a standard assessment of the patient’s performance. On the other hand, the medical usability and validity of the setup and exergames will be tested and assessed by the clinical team of La Tour Hospital over a large set of voluntary patients. Both teams will also work together to develop metrics to evaluate and adapt the patient’s performance during the sessions, thanks to the extended capabilities of the motion tracking system with respect to the data available in conventional physical therapy.

Partners

Artanim
Conception of the exergames and of a platform adapted to perform them, development of a set of medical scores adapted to the extended capabilities of the motion tracking system

La Tour Hospital – Division of Orthopedic and Trauma Surgery
Clinical tests, co-design of new physical evaluation metrics adapted to the motion tracking capabilities

Related Publications

Mancuso M, Lädermann A, Schmid Dorbierer Y, Seurot A, Charbonnier C. Can the Combination of Motion Tracking and Virtual Reality Make Shoulder Rehabilitation Entertaining?, Multidisciplinary Biomechanics Journal, 3:47-49, 2026.
PDF

Mancuso M, Charbonnier C. Technical evaluation of the fidelity of the HTC Vive for upper limb tracking, 42nd International Conference on Biomechanics in Sports, Salzburg, Austria, July 2024.
PDF

 

HoloMed

HoloMed

by | Feb 12, 2018

HoloMed

AR tools for medicine

Project Info

Start date:
September 2016

End date:
February 2018

Funding:
La Tour Hospital / Hirslanden Clinique La Colline

Coordinator:
Artanim

Back

Summary

In medicine, augmented reality (AR) technology can offer added value over traditional education, assessment and interventional approaches. By combining our expertise in motion capture, 3D anatomical modelling and AR technology, new solutions can be proposed to improve medical treatment.

In this project, we developed an anatomical see-through tool to visualize and analyze patient’s anatomy in real time and in motion for applications in sports medicine and rehabilitation. This tool allows healthcare professionals to visualize joint kinematics, where the bones are accurately rendered as a holographic overlay on the subject (like an X-ray vision) and in real-time as the subject performs the movement.

Partners

Artanim
Development of AR solutions

La Tour Hospital – Division of Orthopedic and Trauma Surgery
Clinical tests

Hirslanden Clinique La Colline – Division of Orthopedic and Trauma Surgery
Clinical tests

Related Publications

Elias de Oliveira M, Galvan Debarba H, Lädermann A, Chagué S, Charbonnier C. A Hand-Eye Calibration Method for Augmented Reality Applied to Computer-Assisted Orthopedic Surgery, Int J Med Robot Comput Assist Surg, 15(2):e1969, 2019.
PDF

Elias de Oliveira M, Galvan Debarba H, Lädermann A, Chagué S, Charbonnier C. Réalité augmentée en chirurgie orthopédique. Mythe ou réalité ?, Congrès de la Société Francophone d’Arthroscopie 2018, Paris, France, Revue de chirurgie orthopédique et traumatologique, 104(8):S110, November 2018.
PDF

Galvan Debarba H, Elias de Oliveira M, Lädermann A, Chagué S, Charbonnier C. Augmented Reality Visualization of Joint Movements for Rehabilitation and Sports Medicine, 20th Symposium on Virtual and Augmented Reality (SVR), Foz Do Iguaçu, Brazil, October 2018.
PDF

Galvan Debarba H, Elias de Oliveira M, Lädermann A, Chagué S, Charbonnier C. Augmented Reality Visualization of Joint Movements for Physical Examination and Rehabilitation, Proc. 2018 IEEE Virtual Reality, IEEE Comput. Soc, March 2018.
PDF

Galvan Debarba H, Elias de Oliveira M, Lädermann A, Chagué S, Charbonnier C. Tracking a Consumer HMD with a Third Party Motion Capture System, Proc. 2018 IEEE Virtual Reality, IEEE Comput. Soc, March 2018.
PDF

Elias de Oliveira M, Galvan Debarba H, Lädermann A, Chagué S, Charbonnier C. Development of a Hand-Eye Calibration Method for Augmented Reality Applied to Computer-Assisted Orthopedic Surgery, 32nd International Congress of Computer Assisted Radiology and Surgery, Heidelberg, Germany, Int J CARS, June 2018.
PDF