This project captures the quiddity of a vibraphonist embedded in improvised music by presenting the music from mallets’ perspectives.
Music improvisation allows musicians to freely express and communicate their feelings and stories at the moment. For a vibraphonist, two pairs of mallets held by their left and right hands are the main vehicle of their feelings and thoughts. Through deliberate controls of the two pairs of mallets, a vibraphonist forms harmony and composes music.
This project delves into the quiddity of a vibraphonist in the process of improvisation by shifting the perspective to these two pairs of mallets. The system tracks the movements of two mallets and fixes the focus on each pair of mallets. This project is inspired by Evan Roth’s White Glove Tracking.
To be more specific, the system works in the following steps. The steps are visualized in the video below.
- The software built with OpenFrameworks tracks red and yellow blobs separately as shown in the picture below. In the picture, you can see white bounding boxes with green labels around the yellow mallet heads. The software uses ofxOpenCv, ofxCv, and ofxGui packages for tracking and parameter adjustment.
- Based on the positions of two mallet heads, the software rotates, scales, and translates the video so that the pair of mallets is always centered with the same distance.
- I used Adobe Premiere Pro to composite the two red mallet-centered and yellow mallet-centered videos.
- For additional exploration, I tried out different visual effects in an attempt to simulate the improvisation process inside the vibraphonist’s head — for example, splitting the screen into left and right cuts, overlaying the two sides, and flipping the videos upside down as if we’re looking through the vibraphonist’s eyes.
This mallet-centered view of improvisational music reveals the quiddity of the vibraphonist more than still photos or normal videos.