Project 1: Video Sonification

For this project, I decided to take camera input and use different characteristics of the video to generate sound. I split the input into 4 matrices, and used different characteristics of each of the 4 matrices and mapped them to 4 different sounds. The characteristics I used included the hue, saturation, and lightness values as well as the RGB values of the images. I also wanted to add a visual component, so I used jit.sobel for edge detection on all 4 parts, with 2 of the parts remaining as the output of the edge detection, and the other 2 parts having the edge detection output blended with the video with the brightness and saturation varying based on the original video input. I then stitched the parts back together to get a cool distorted image to go along with the eerie sound that is generated.

Here is my patch:

Here is a video that shows the patch working: