Assignment 4 – color + mousestate

For this assignment I decided to experiment with the mousestate project. Based on a meshy patch we made in class and the freeze frame patch from Jean-Francois Charles’ tutorial, I embedded two pfft~ subpatches.

The first one called “hw4-pfft” is in charge of creating input matrix for the mesh object and a single float value from amplitude bin for color shifting. As shown below, the value is scaled to 0.0 – 1.0 before entering jit.gl.gridshape, and the specific range could be changed depending on the amplitudes of different audio inputs. Note that the two other float values (saturation and grayscale) in the message box can also be altered based on changing values like the first one, but since rapidly changing saturation or grayscale don’t really create drastic effects, I decided to hardcode them after experimenting with various values.

I also incorporated the mousestate object to track the speed of mouse movement, which decides how often the audio “freezes”. The last two outlets of mousestate indicate the mouse’s horizontal and vertical movement during the 50 ms interval. The faster it moves, the more dramatic the freezing effect.

This is the first audio input, Sapokanikan by Joanna Newsom:

 

Output: (Mouse movements are easier to see in full-screen mode)

 

The second input audio is a woman whispering in Swedish saying “never tell this secret to anyone else, not your best friend, not your parents” that I found on freesound.org:

Since the amplitude is relatively constant, the color of the mesh object does not change much, but the freezing effect is especially interesting with (creepy) speeches:

 

My main patch:

Freeze subpatch:

fft subpatch: