New venues for capturing facial performance

New venues for capturing facial performance

We will soon start shooting cinematic content to be used for showcasing the technology developed by the VR-Together consortium. In this post, we bring some of the production effort developed at Artanim, which is currently exploring the use of Apple’s iPhone X face tracking technology in the production pipeline of 3D animations.

Facial mocap rig

The photos below show the iPhone X holding rig, and an early version of the face tracking recording tool that was developed by Artanim. The tool integrates with full body and hands motion capture technology from Vicon to allow the simultaneous recording of body, hands and face performance from multiple actors.

With the recent surge of consumer virtual reality, interest for motion capture has dramatically increased. The iPhone X and ARKit SDK from Apple integrate depth sensing and facial animation technologies, and are a good example of this trend. Apple’s effort to integrate advanced face tracking to their mobile devices may be related to the recent acquisition of PrimeSense and FaceShift. The former was involved in the development of the technology powering the first Kinect back in 2011, the latter is recognized for their face tracking technology, which is briefly showcased in the making of Star Wars: The Force Awakens trailer. These are exciting times, when advanced motion tracking technologies are becoming ubiquitous in our life.

Image from the iPhone X keynote presentation from Apple

3DIM @ Montreux Jazz Festival

3DIM @ Montreux Jazz Festival

3D In Motion (3DIM) – our experimental setup of capture, visualization and sonification of movements in real time – was presented at the Montreux Jazz Festival on the 12th of July 2014 during an one hour workshop. For this occasion, a special sound design was developed by Alain Renaud from MINTLab. A mapping of different artists’ audio tracks recorded during their visit to the festival was performed. The tracks were generously provided by the Montreux Jazz Lab of EPFL.

Moreover, several improvements in the 3DIM application were achieved. For instance, an iPad App was implemented to easily switch between the sound design scenarios, a prompter was added to provide instructions to the user and an OSC connection between the sonification and graphical applications was programmed to control visual feedback from sound events.

Again, the feedback from the public was positive. We look forward to developing a first live performance using this system.

More information about the 3DIM project here.

A demo video of the setup including the sonification part will arrive soon. Stay in touch!

Montreux Jazz Festival demo Montreux Jazz Festival demo

3DIM @ Electron Festival

3DIM @ Electron Festival

In collaboration with Alain Renaud from MINTLab, we gave on the 20th of April a workshop at Electron Festival in Geneva. At this occasion, we presented the 3D In Motion (3DIM) experimental setup of capture, visualization and sonification of movements in real time. About twenty people participated to the workshop, where they had the opportunity to test the system and to learn more about the underlying techniques.

It was the first time 3DIM was officially presented to the public and the feedback was positive. This gave us a good basis for further developing the system both in terms of visualization and musicality. The next demonstration will take place in June at the New Interfaces for Musical Expression (NIME) Conference in London.

More information about the 3DIM project here.

Electron Festival demo Electron Festival demo

Grand Prix d’Horlogerie de Genève 2011

Grand Prix d’Horlogerie de Genève 2011

Artanim participated in the Grand Prix d’Horlogerie de Genève 2011 which rewards the finest watch creations of the year. The ceremony was held at the Grand Théâtre de Genève on Saturday 19th of November 2011. Simply the best Swiss watchmakers were present, as well as the President of the Confederation – Micheline Calmy-Rey – the Director of the United Nations in Geneva and Ambassadors.

Under the theme “Time in the cinema”, artanim created in collaboration with Move for Art a show combining dance and 3D animation, a leap into the future marked by the inevitable epic sci-fi film “Star Wars” created by George Lucas. Darth Vader, the head of the imperial army, was embodied on stage by a dancer (Manuel Vignoulle). Its 3D clone, as well as the imperial troops were projected on big screens behind him and animated by the rhythm of his motion.

GPHG  Artanim at GPHG