Mendelssohn Effektorium: How to Control an Orchestra (with Live)

The road to becoming a conductor is a long and hard one, requiring years of dedication and a glut of musical knowledge. It’s a possibility that’s beyond most music fans. What is it like to be at the steering wheel of a super-sized orchestra? How does it feel to have your musical wishes played out by a group of virtuosos under your direct control? The closest most will get is a front row seat in the concert hall.

Enter technology, and the Berlin-based collaboration between Aconica, WHITEvoid and Bertron Schwarz Frey. The trio has developed an interactive audio-visual installation that allows the public to manipulate the micro and macro elements of a virtual orchestra. Dubbed the Effektorium, it’s housed in Leipzig’s Mendelssohn-Bartholdy Museum. Check out the video below, and watch how using a baton and a digital score, would-be conductors can captain the virtual orchestra through Mendelssohn’s works with the gestures and inflections of a real maestro.

So, how does it work? Behind the scenes, a-two computer system communicates OSC messages transmitted through an ethernet cable. One computer runs the motion tracking software LEAP Motion, (which records physical motion and transforms the information into a digital signal), as well as the touchscreen and an array of custom speakers. The other computer uses Live to house audio files and a suite of convolution reverb plugins, in addition to Warping the pieces to the conductor’s tempo. On the same computer is a complex Max for Live patch which syncs the whole thing together. Contact with the touchscreen sends OSC messages to the Max patch, which allows conductors to jump through the score at will, with zero latency or jitter.

The LEAP software captures the speed of the conductor’s gestures to control the tempo, while the touchscreen handles a myriad of variables: the volume of the woodwinds, brass, strings, chorus or percussion, the character of the reverb, the orchestra’s position in the score, even whether the ensemble is tuned to the historical or modern standard of tuning (that’s 430hz and 443hz respectively by the way).

Conductors can also control the mood of the installation by altering the hue and intensity of LED light housed in the upright speaker array. Additionally, each set of instruments is sent to individual speakers, allowing listeners to isolate elements of the composition, one segment at a time.

You can read more about the project on the Aconica website, and visit the installation at the Mendelssohn-Bartholdy Museum in Leipzig.