|
Integrated Multimodal Score-following Environment (IMuSE):
The Integrated Multimodal Score-following Environment (IMuSE) is a research project funded by
the Social Sciences and Humanities Research Council (SSHRC) starting in 2009.
IMuSE is a software environment based on the NoteAbilityPro music editor and the IIMPE system developed by
Keith Hamel and Bob Pritchard. With IMuSE, score-following is expanded from audio pitch-tracking to also include a variety of
multi-channel gesture data. Live performance gestures can come from any kind of single or multi-channel continuous data from standard controllers,
accelerometers, movement trackers, video trackers, live audio analysis etc. This data is matched against pre-recorded gesture data which has been stored in the
NoteAbilityPro score as multi-channel break-point functions.
Polyphonic pitch-tracking is performed using the antescofo score-following software developed by
Arshia Cont and gesture-tracking is performed using the gf software developed by Frederic Bevilacqua
and Bruno Zamborlin. IMuSE negotiates between tracking data received from multiple sources
and adjusts score playback based on an assessment of which data is most reliable at a given point in the performance.
An example of a NoteAbilityPro score containing the score to be played by a live performer, MaxMSP control messages and gestures recorded from a 3-axis accelerometer.
|
