Cinejack:using live music to control narrative visuals

We present Cinejack, a system for directing narrative video through live musical performance. Cinejack interprets high-level musical content from live instruments and translates it into cinematographic actions such as edits, framings and simulated camera movements. We describe Cinejack's technical development in terms of a novel and highly pragmatic approach to interface design, where the affordances of users' own musical instruments are used as controllers through an interpretive interaction scheme.