Pinocchio

The goal of Pinocchio is to enable children to experience and develop an understanding for classical music in a playful manner by conducting a virtual orchestra. We are developing Pinocchio as a game to be usable in a living room environment as well as in a museum setting. The user can configure the orchestra, which is modeled as a set of individually controllable musicians. Each musician is represented by audio and video streams and can be freely located in 3D space. The orchestra adjusts its tempo and volume to the conductor's gestures. Pinocchio works with several types of batons, differing in tracking method and in gesture recognition algorithms. The long-term goal is the creation of a multi-modal, device independent framework for gesture-based applications which require motoric skills or the control and operation of a complex set of sensors in intelligent house or car driver assistance systems.