Real-time Video-based Character Animation

The ability to animate a 3D virtual character in real-time has great potential in terms of connecting and interacting with an audience or individual user, for example allowing a virtual character to answer questions from audience members or make spontaneous comments about a show, or by allowing a gamer to represent themselves more effectively in the game world. Existing professional animation production methods such as digital puppetry, based on complex manual controllers, and optical performance capture, based on reflective markers require specialist equipment, on-set expertise, time consuming setup and are therefore not appropriate for a mass market consumer context, and their relative expense limits their use in real-time professional media production. Recently systems have been demonstrated that reduce these requirements somewhat [Weise et al. 2011], but still require emerging specialist consumer grade 3D capture devices. We have developed a new method of real-time character animation that requires no expertise and uses widely available consumergrade camera equipment as input and that can be deployed on standard hardware such as laptops and mobile devices.