Real Virtuality

Real Virtuality

by | Sep 23, 2021

Real Virtuality

A multi-user immersive VR platform combining full-body motion capture with physical space and haptics.

Project Info

Start date

April 2015

End date

December 2025

Funding

Artanim

Coordinator

Artanim

Back to Projects

Summary

Real Virtuality is a multi-user immersive VR platform invented and developed by Artanim between 2015 and 2025. The technology powered the location-based experiences of Dreamscape Immersive and was later extended to experiential learning through Dreamscape Learn.

The platform combines a shared 3D environment experienced through VR headsets with a physical stage incorporating real objects and haptic elements. Users are tracked in real time and represented by full-body avatars whose movements reproduce their physical actions inside the virtual world.

Up to eight participants can move freely together, see and interact with one another, manipulate physical and virtual objects, and explore the same shared environment. By aligning the physical and digital spaces, Real Virtuality turns the user’s body into the interface between the real and virtual worlds.

The platform combines several key features:

  • Wireless: freedom of movement across large physical spaces.
  • Social: shared multi-user experiences within the same environment or across connected spaces.
  • Accurate: high-precision tracking of users and physical objects.
  • Real-time: low-latency interaction between physical movement and the virtual environment.
  • Flexible: an SDK enabling content creators to develop experiences specifically for the platform.

Real Virtuality was used across entertainment, cultural and educational applications. Artanim also used the platform to create VR_I, an immersive choreographic work; Geneva 1850, a historical journey into 19th-century Geneva; Escalade: The Darkest Night, an immersive reconstruction of the events of December 1602; and Debug Mode, a free-flying multiplayer VR experience combining fast-paced gameplay, storytelling and humor.

Related Publications

Chagué S, Charbonnier C. Real Virtuality: A Multi-User Immersive Platform Connecting Real and Virtual Worlds, VRIC 2016 Virtual Reality International Conference – Laval Virtual, Laval, France, ACM New York, NY, USA, 1–3, March, 2016.
PDF · DOI

Chagué S, Charbonnier C. Digital Cloning for an Increased Feeling of Presence in Collaborative Virtual Reality Environments, Proc. of 6th Int. Conf. on 3D Body Scanning Technologies, Lugano, Switzerland, October, 2015.
PDF · DOI

Charbonnier C, Trouche V. Real Virtuality: Perspectives Offered by the Combination of Virtual Reality Headsets and Motion Capture, White Paper, August, 2015.
PDF · DOI

Awards and Recognition

Char4VR

Char4VR

by | May 22, 2021

Char4VR

A research project exploring intelligent virtual characters for interactive narrative experiences in VR.

Project Info

Start date

September 2020

End date

December 2023

Funding

Artanim

Coordinator

Artanim

Back to Projects

Summary

Virtual reality opens the possibility of creating narrative experiences that combine the richness of film or theater with direct interaction between users and autonomous virtual characters. Rather than following a fully predetermined sequence, participants can observe a story unfold around them, interact with its characters and potentially influence parts of the narrative.

A key challenge is that conventional character animation techniques, largely developed for video games, do not always support the subtle and multimodal interactions that users naturally expect in immersive VR. Virtual characters need to respond convincingly to a user’s presence, movements and behavior while remaining consistent with the narrative.

The project explores new approaches to interactive character animation aimed at creating more engaging and believable virtual characters. It combines methods from computer graphics, machine learning and cognitive psychology to investigate how autonomous characters can move, react and interact naturally with users within immersive narrative environments.

Related Publications

Llobera J, Charbonnier C. Physics-based character animation and human motor control, Phys Life Rev, 46:190–219, 2023.
PDF · DOI

Llobera J, Jacquat V, Calabrese C, Charbonnier C. Playing the mirror game in virtual reality with an autonomous character, Sci Rep, 12:21329, 2022.
PDF · DOI

Llobera J, Charbonnier C. Physics-based character animation for Virtual Reality, Open Access Tools and Libraries for Virtual Reality, IEEE VR Workshop, 2022 Best Open Source Tool Award, March, 2022.
PDF · DOI

Llobera J, Booth J, Charbonnier C. Physics-based character animation controllers for videogame and virtual reality production, 14th ACM SIGGRAPH Conference on Motion, Interaction and Games, November, 2021.
PDF

Llobera J, Booth J, Charbonnier C. New Techniques in Interactive Character Animation, SIGGRAPH ’21: short course, ACM, New York, NY, USA, 16:1–6, August, 2021.
PDF · DOI

Llobera J, Charbonnier C. Interactive Characters for Virtual Reality Stories, ACM International Conference on Interactive Media Experiences (IMX 2021), ACM, New York, NY, USA, 322–325, June, 2021.
PDF · DOI

Real-Time Markerless Motion Capture

Real-Time Markerless Motion Capture

by | Apr 17, 2021

Real-Time Markerless Motion Capture

A machine-learning-based system for real-time multi-user motion capture without markers.

Project Info

Start date

September 2020

End date

September 2023

Funding

Artanim & Vicon

Coordinator

Artanim

Back to Projects

Summary

Real-Time Markerless Motion Capture explores how machine learning can remove the need for physical markers in location-based virtual reality (LBVR) while preserving the speed and accuracy required for immersive multi-user experiences.

Traditional LBVR motion capture systems rely on active or passive infrared markers attached to users. While this provides stable and low-latency tracking, preparing participants with markers is time-consuming and creates additional maintenance and operational constraints.

The project develops a markerless pipeline optimized for real-time performance. Camera images are processed at 60 frames per second, 2D body poses are estimated independently in each view, and the results are combined into full 3D skeletons. To meet the strict latency requirements of immersive VR, the pipeline is optimized across modern CPU and GPU architectures, balancing tracking accuracy with computational speed.

A second major challenge is the creation of sufficiently large and accurately annotated datasets for training machine-learning models. Rather than relying only on existing datasets, Artanim developed its Synthetic Factory to generate diverse virtual populations with variations in body shape, age, appearance, clothing, footwear, hairstyles and movement. Because the underlying virtual skeleton is known exactly, these synthetic subjects provide reliable ground-truth data for training and evaluation.

The project was developed in collaboration with Vicon. The first results were publicly demonstrated at SIGGRAPH 2023, where six participants were tracked simultaneously in real time without markers during The Clockwork Forest, a Dreamscape location-based VR experience created in partnership with Audemars Piguet. The showcase received a CGW Silver Edge Award for technological innovation.

Partners

Artanim
Definition of VR requirements, evaluation of machine-learning-based tracking algorithms, development of the Synthetic Factory, and implementation and fine-tuning of the multimodal tracking solution.

Vicon
Hardware development, implementation of machine-learning-based tracking algorithms, and real-time pose solving.

Related Publications

Nagorny P, Kevelham B, Chagué S, Charbonnier C. A Comprehensive Review of Real-Time Multi-View Multi-Person Markerless Motion Capture, ACM Comput Surv, 58(3):1–34, 2025.
PDF · DOI

VR+4CAD

VR+4CAD

by | May 8, 2020

VR+4CAD

A VR research project connecting CAD design, markerless interaction and user feedback.

Project Info

Start date

May 2020

End date

November 2021

Funding

Innosuisse – Project No. 42975.1 IP-ICT

Coordinator

Artanim

Website

VR+4CAD project website

Back to Projects

Summary

VR+4CAD addresses several barriers limiting the adoption of virtual reality in manufacturing and design: incomplete interoperability between VR and CAD workflows, the friction involved in entering and interacting with virtual environments, and the limited feedback available for subsequent design analysis.

The project investigates how CAD-authored designs can be automatically converted and adapted for direct human interaction in VR. An experimental markerless motion capture system allows users to interact naturally with virtual prototypes without wearing dedicated tracking devices.

Motion data collected during each session is analyzed using activity recognition techniques to generate implicit feedback about user behavior and interaction. This information is combined with explicit user feedback and returned to the CAD operator, supporting the next iteration of the design process.

Partners

Caecilia Charbonnier – Artanim
Markerless motion capture, VR activity annotation and development of the interactive VR tools.

Silvia Giordano – University of Applied Sciences and Arts of Southern Switzerland (SUPSI)
CAD/VR interoperability and human activity recognition.

VR-Together

VR-Together

by | Jun 15, 2019

VR-Together

A Horizon 2020 project developing photorealistic social VR for shared immersive experiences.

Project Info

Start date

October 2017

End date

December 2020

Funding

European Commission – Horizon 2020 (Grant Agreement No. 762111)

Website

CORDIS project page

Coordinator

Fundació i2CAT

Back to Projects

Summary

VR-Together explores how photorealistic immersive content can enable compelling social virtual reality experiences. The project develops an end-to-end pipeline combining capture, encoding, network delivery and rendering technologies to bring remote participants together in shared virtual environments.

A key objective is to make high-quality social VR practical and scalable using a combination of advanced technologies and commercially available components. The platform supports both live and interactive immersive content and is evaluated through three pilot applications covering different production and usage scenarios.

The project also develops methods for evaluating social presence and user experience, together with quantitative benchmarks for assessing the performance of immersive media production and delivery pipelines.

Within VR-Together, Artanim contributed to content production for the three pilots, combining VR with offline and real-time motion capture. Artanim also developed tools for immersive media production and participated in the evaluation of the resulting social VR experiences.

Partners

Fundació i2CAT (Spain)

Netherlands Organisation for Applied Scientific Research – TNO (The Netherlands)

Centrum Wiskunde & Informatica – CWI (The Netherlands)

Centre for Research and Technology Hellas – CERTH (Greece)

Viaccess-Orca (France)

Entropy Studio (Spain)

Motion Spell (France)

Artanim (Switzerland)

Related Publications

Galvan Debarba H, Montagud M, Chagué S, Lajara J, Lacosta I, Fernandez Langa S, Charbonnier C. Content Format and Quality of Experience in Virtual Reality, Multimed Tools Appl, 83:46481–46506, 2024.
PDF · DOI

Galvan Debarba H, Chagué S, Charbonnier C. On the Plausibility of Virtual Body Animation Features in Virtual Reality, IEEE Trans Vis Comput Graph, 28(4):1880–1893, 2022.
PDF · DOI

Revilla A, Zamarvide S, Lacosta I, Perez F, Lajara J, Kevelham B, Juillard V, Rochat B, Drocco M, Devaud N, Barbeau O, Charbonnier C, de Lange P, Li J, Mei Y, Lawicka K, Jansen J, Reimat N, Subramanyam S, Cesar P. A Collaborative VR Murder Mystery using Photorealistic User Representations, Proc. IEEE Conf. on Virtual Reality and 3D User Interfaces (VRW 2021), IEEE, pp. 766, March, 2021.
PDF · DOI

Chatzitofis A, Saroglou L, Boutis P, Drakoulis P, Zioulis N, Subramanyam S, Kevelham B, Charbonnier C, Cesar P, Zarpalas D, Kollias S, Daras P. HUMAN4D: A Human-Centric Multimodal Dataset for Motions & Immersive Media, IEEE Access, 8:176241–176262, 2020.
PDF · DOI

De Simone F, Li J, Galvan Debarba H, El Ali A, Gunkel S, Cesar P. Watching videos together in social Virtual Reality: an experimental study on user’s QoE, Proc. 2019 IEEE Virtual Reality, IEEE, 890–891, March, 2019.
PDF · DOI