Char4VR

Char4VR

Char4VR

Interactive characters for VR experiences

Project Info

Start date:
September 2020

End date:
August 2023

Funding:

Coordinator:
Artanim

Summary

Virtual Reality (VR) opens the possibility to develop a new kind of content format which, on one hand, is experienced as a rich narrative, just as movies or theater plays are, while on the other hand, it allows interaction with autonomous virtual characters. The experience would feel like having different autonomous characters unfold a consistent story plot, just as a movie or a theater play does, within an interactive VR simulation. Players could choose to participate in the events and affect some parts of the story, or just watch how the plot unfolds around them.

The main challenge to realize this vision is that current techniques for interactive character animation are designed for video games, and do not offer the subtle multi-modal interaction that VR users spontaneously expect.  The main goal of this project is to explore different techniques for interactive character animation that help creates interactive characters that can engage in a more compelling way with players. To achieve this goal, we will use a combination of research techniques derived from computer graphics, machine learning and cognitive psychology.

Related Publications

Llobera J, Charbonnier C. Interactive Characters for Virtual Reality Stories, ACM International Conference on Interactive Media Experiences (IMX \’21), ACM, New York, NY, USA, June 2021.
PDF

Real Virtuality

Real Virtuality

Real Virtuality

Immersive platform

Project Info

Start date:
April 2015

End date:
December 2022

Funding:

Coordinator:
Artanim

Summary

Artanim invented and develops since 2015 the VR technology driving Dreamscape Immersive. This multi-user immersive platform combines a 3D environment – which can be seen and heard through a VR headset – with a real-life stage set incorporating haptic feedback elements.

The user’s movements are sparsely captured in real time and translated into a full-body animation thanks to a deep understanding of body mechanics. Contrary to other static position VR systems, the platform allows up to eight users to truly feel immersed in a VR scene. They are rendered as characters (avatars) inside a computer generated world where they can move physically, interact with objects and other players, and truly experience worlds previously accessible only in their imagination. The bodies of the users thus become the interface between the real and virtual worlds.

The platform combines the following features:

  • Wireless: complete freedom of movement across large spaces.
  • Social: interactive multi-user experiences within a shared environment or across connected pods.
  • Accurate: full-body and physical objects tracking with less than 1 millimeter deviation.
  • Real-time: zero discernible lag eliminating most concerns of motion sickness.
  • Flexible: SDK allowing content creators to easily create experiences for this particular platform.

This platform is leveraged by Dreamscape Immersive through their worldwide location-based VR entertainment centers, as well as through educational and training programs and other enterprise solutions.

Related Publications

Chagué S, Charbonnier C. Real Virtuality: A Multi-User Immersive Platform Connecting Real and Virtual Worlds, VRIC 2016 Virtual Reality International Conference – Laval Virtual, Laval, France, ACM New York, NY, USA, March 2016.
PDF
 
Chagué S, Charbonnier C. Digital Cloning for an Increased Feeling of Presence in Collaborative Virtual Reality Environments, Proc. of 6th Int. Conf. on 3D Body Scanning Technologies, Lugano, Switzerland, October 2015.
PDF

Charbonnier C, Trouche V. Real Virtuality: Perspective offered by the Combination of Virtual Reality Headsets and Motion Capture, White Paper, August 2015.
PDF

Awards and Recognitions

VR-Together

VR-Together

VR-Together

Social VR experiences

Project Info

Start date:
October 2017

End date:
December 2020

Funding:
EU H2020

Coordinator:
Fundació i2CAT

Website:
http://vrtogether.eu/

Summary

VR-Together offers new ground-breaking virtual reality (VR) experiences based on social photorealistic immersive content. For this purpose, this project developed and assembled an end-to-end pipeline integrating state-of-the-art technologies and off-the-shelf components. The challenge of VR-Together was to create photorealistic truly social VR experiences in a cost effective manner. Immersive media production and delivery through innovative capture, encoding, delivery and rendering technologies. The project demonstrated the scalability of its approach for production and delivery of immersive content across 3 pilots. It introduced new methods for social VR evaluation and quantitative platform benchmarking for both live and interactive content production, thus providing production and delivery solutions with significant commercial value.

In this project, Artanim worked on content production for the 3 pilots combining VR and offline/real time motion capture and developed new tools for immersive media production. We also participated in the evaluation of the developed social VR experiences.

Partners

Fundació i2CAT (Spain)

Netherlands Organisation for Applied Scientific Research (The Netherlands)

Centrum Wiskunde & Informatica (The Netherlands)

Centre for Research & Technology Hellas (Greece)

Viaccess-Orca (France)

Entropy Studio (Spain)

Motion Spell (France)

Artanim

Related Publications

Chatzitofis A, Saroglou L, Boutis P, Drakoulis P, Zioulis N, Subramanyam S, Kevelham B, Charbonnier C, Cesar P, Zarpalas D, Kollias S, Daras P. HUMAN4D: A Human-Centric Multimodal Dataset for Motions & Immersive Media, IEEE Access, 8:176241-176262, 2020.
PDF

Galvan Debarba H, Chagué S, Charbonnier C. On the Plausibility of Virtual Body AnimationFeatures in Virtual Reality, IEEE Trans Vis Comput Graph, In Press, 2020.
PDF

De Simone F, Li J, Galvan Debarba H, El Ali A, Gunkel S, Cesar P. Watching videos together in social Virtual Reality: an experimental study on user’s QoE, Proc. 2019 IEEE Virtual Reality, IEEE Comput. Soc, March 2019.
PDF

HoloMed

HoloMed

HoloMed

AR tools for medicine

Project Info

Start date:
September 2016

End date:
February 2018

Funding:
La Tour Hospital / Hirslanden Clinique La Colline

Coordinator:
Artanim

Summary

In medicine, augmented reality (AR) technology can offer added value over traditional education, assessment and interventional approaches. By combining our expertise in motion capture, 3D anatomical modelling and AR technology, new solutions can be proposed to improve medical treatment.

In this project, we developed an anatomical see-through tool to visualize and analyze patient’s anatomy in real time and in motion for applications in sports medicine and rehabilitation. This tool allows healthcare professionals to visualize joint kinematics, where the bones are accurately rendered as a holographic overlay on the subject (like an X-ray vision) and in real-time as the subject performs the movement.

Partners

Artanim
Development of AR solutions

La Tour Hospital – Division of Orthopedic and Trauma Surgery
Clinical tests

Hirslanden Clinique La Colline – Division of Orthopedic and Trauma Surgery
Clinical tests

Related Publications

Elias de Oliveira M, Galvan Debarba H, Lädermann A, Chagué S, Charbonnier C. A Hand-Eye Calibration Method for Augmented Reality Applied to Computer-Assisted Orthopedic Surgery, Int J Med Robot Comput Assist Surg, 15(2):e1969, 2019.
PDF

Elias de Oliveira M, Galvan Debarba H, Lädermann A, Chagué S, Charbonnier C. Réalité augmentée en chirurgie orthopédique. Mythe ou réalité ?, Congrès de la Société Francophone d’Arthroscopie 2018, Paris, France, Revue de chirurgie orthopédique et traumatologique, 104(8):S110, November 2018.
PDF

Galvan Debarba H, Elias de Oliveira M, Lädermann A, Chagué S, Charbonnier C. Augmented Reality Visualization of Joint Movements for Rehabilitation and Sports Medicine, 20th Symposium on Virtual and Augmented Reality (SVR), Foz Do Iguaçu, Brazil, October 2018.
PDF

Galvan Debarba H, Elias de Oliveira M, Lädermann A, Chagué S, Charbonnier C. Augmented Reality Visualization of Joint Movements for Physical Examination and Rehabilitation, Proc. 2018 IEEE Virtual Reality, IEEE Comput. Soc, March 2018.
PDF

Galvan Debarba H, Elias de Oliveira M, Lädermann A, Chagué S, Charbonnier C. Tracking a Consumer HMD with a Third Party Motion Capture System, Proc. 2018 IEEE Virtual Reality, IEEE Comput. Soc, March 2018.
PDF

Elias de Oliveira M, Galvan Debarba H, Lädermann A, Chagué S, Charbonnier C. Development of a Hand-Eye Calibration Method for Augmented Reality Applied to Computer-Assisted Orthopedic Surgery, 32nd International Congress of Computer Assisted Radiology and Surgery, Heidelberg, Germany, Int J CARS, June 2018.
PDF

Taylormatic

Taylormatic

Taylormatic

A multi-platform solution

Project Info

Start date:
December 2012

End date:
October 2014

Funding:
NoBrandSolution

Coordinator:
NoBrandSolution

Website:
http://www.nobrandsolution.com/

Summary

The goal of this project was to provide a multi-platform product presentation and customization solution. The Taylormatic platform allows designers and producers to present their customizable products in a virtual yet visually realistic manner to potential clients, allowing them to manufacture products as required. At the same time it provides users with the freedom to configure products to their liking, while presenting them with a visually convincing result.

As part of this project, we worked on the research and development of a Kinect-based gesture recognition solution, as well as a multi-Kinect body scanning solution. The body scanner allows for the accurate size-estimation of fully dressed customers, for products where a size recommendation needs to be made.

Taking into account the full 3D asset production pipeline from initial design to final 3D content, we furthermore designed and developed asset management plugins for Unity which allow for the management and export of 3D assets and their meta data. Based on this data, the applications we developed ranging from a full in-store virtual mirror to optimized iOS and Android solutions, can be fully customized to support a wide variety of product types and visual identities.

Partners

NoBrandSolution
Concept creation, commercialization, asset design and creation

Artanim
Body scanning solution, gesture recognition, application development and asset management