Final Project – Computer Vision Sampler

The Team
Kaitlin Schaer & Marc-Daniel Julien

Introduction
In our computer vision music sampler, we sought to create a physical interface that would allow the user to explore a song in a new way. Using computer vision software, a webcam, a projector, and interactive play surface, our sampler allows the user to move through a piece of music and make it their own.

Piece by piece, the user places the acrylic pieces on the play surface. As in a traditional sampler, clips of music will play according to the objects’ positioning. However instead of using a typical rectilinear layout, we have chosen a polar system, to be both stage-like and reminiscent of a record player.

In this prototype we are working with nine song clips which are distinguished by the color and geometry of our pieces. We have chosen to explore the song Come Together by the Beatles for its strong beats, bassline, and vocal hooks, as well as its recognizability.

As a finished product, we imagine that the user would be able to draw from a larger number of clips, from any number of different songs.

Video

Technical Notes

Software

Github Link

Construction

The physical aspects of the computer vision sampler were realized mostly through computer aided design processes. Modeled in rhino, and laser cut from 1/4 in MDF, the structure was designed with several things in mind: accessibility from multiple angles, the need to mount both a camera and projector, and the need to easily adjust the positioning of said equipment. The shapes used to control the samples, and the labels noting the different instruments available, are colored acrylic plexiglass.

Additional Photos