Author: daïa

Audio Decomposition(Final Documentation)

This project was such a great learning experience for me even though it didn’t turn out how I expected. I made two master max patches; one involving all of the steps for receiving and sending information to the serial port of the arduino and one for transforming information received into a legible signal full of various oscillators.  The idea remained the same from the initial writeup of identifying key parts of the voice and decomposing them into various signals able to be sent out to a synthesizer. 

 The initial idea – have the ability to talk to or sing with the synth to control its output. I wanted to do this using CV or control voltage. By sending the Arp 2600 (synthesizer) a signal between zero and five volts the synth can respond almost immediately to whatever input I send that to.

The first thing is translating information from my microphone into digital information that can be further processed. I decided to go with using the teachable machine route and training a model on my voice to give interactivity but also a classifier to go by. However, teachable machine cannot be loaded into max given some package compatibility issues. So, with Golan’s help I was able to take my teachable machine and load it into a p5 sketch, then send the information (classifier and confidence level of the sound) to a bridge (node.js) over osc to receive all of this information and convert it in any way I find fit using max msp.

In my first patch, the idea was to send out a series of notes, similar to a sequencer once a sound was identified, but I only got so far as to send out a different sound depending on the confidence of the sound as a result of the arduino having issues receiving data. 

But first the signals have to be converted to a format that the synth can read.  The synth can receive digital signals that aren’t cv data, but they can’t control the arp in the same way. I wanted to use a combination of both to have a sort of consistency in the way they worked together. Kind of like one of the oscillators being part of the voice signal and the rest of the quality information being used to control the pitch or envelope of the sound. The littlebits have a cv bit that allow me to simply send the littlebits arduino a signal between 0 and 255 and then convert those numbers to cv data.

The edited patch instead of going to the arduino is sent to various preset oscillators and changes sound depending on the classifier it is given. 

Both patches together are shown below

Below I have some video documentation of how things work:

https://vimeo.com/1040094594

https://vimeo.com/1040096752

https://vimeo.com/1040484008

Project Proposal (Final) Synth Play

Pipeline/s

Overall (Edited) Pipeline: 

Mic – computer – little bits (arduino + cv) – audio amplifier – arp

Inside the Computer: Teachable machine to p5.js to max msp out to the analog pipeline

Themes:


Dealing with Loss – Audio Degeneration

My concept is focusing on the idea of losing someone close to you for whatever reason and I want to make a performance out of this using the arp 2600. The digital (capture ) component of this is focusing on removing various phonemes from my voice in real time to either single out or remove completely the specified sounds.

First step – get the system working to use cv to send out different qualities of the signal as a cv to control the arp. This is working 

https://drive.google.com/file/d/1XrmtC7oAI06D0D4_hy1Dk09lZ60820F4/view?usp=drive_link

Training using teachable machine – finding it’s quirks – consistency in volume I think vocal dynamics isn’t such a great way to train this model – prediction is that it becomes confused

 

  • Current headache – Max has odd syntax quirks that aren’t currently compatible with the arduino syntax <numbers>, however they definitely want to talk to each other.There is some conversion edit I have to make. When I send information i get an error which ends the process of sending the numbers, but I get a blink confirming that my setup is almost correct. – just found out / solved it !!!

Next steps – combining the teachable machine that is in p5.js(thanks golan), into max, then getting an output – transforming that output and hence sending it out to the arp. Performance (yay)

Final Proposal – Synth Play (Edited)

Equipment

Lots of cables, arp 2600, amplifier, gigaport, Interface? Max Msp

First step – get the system working to use cv to send out different qualities of the signal as a cv to control the arp.

There may be interactivity involved eventually in how the sound is transformed before it reaches the arp, but right now the focus is to figure out how to send output from my computer and transform it using the gigaport to send to an amplifier and out to the arp. The arp can then be impacted more remotely and have qualities of a specific sound – seems as though the case study will be my voice.

In a sentence- Learn cv well. I want to become proficient in transforming my own voice and other signals into a control signal for the arp. Maybe make interactivity further from that. I’m thinking having a granular decay might be cool.

 

 

Got the setup to work – but edit to idea.

Loss of people – degeneration of data

 

Main Edit to proposal :

My concept is focusing on the idea of losing someone close to you for whatever reason and I want to make a performance out of this using the arp 2600. The digital (capture ) component of this is focusing on removing various phenomes from my voice in real time to either single out or remove completely the specified sounds.

 

Person In Time: Boatin’

Link:

This project was really fun in a lot of ways and a big challenge in several other ways. I had to drop off many side bits to the project that were unanticipated, but I learned a lot about the pains of mixing so many channels of sound.

Binaural Mic – Needs two microphone xlr inputs.  Plugging into a mixer with a mic and instrument will not suffice as it needs phantom power to go to both inputs

Ambisonic Mic – tends to be on the quieter side

Wind is impossible to avoid when going 40mph on a boat – with windshields for all mics.

Setup includes:

Equipment:

Presonus Interface, LOTS OF TAPE, binaural microphone, ambisonic microphone, windcaps , two usbc cables and a dongle

Software: Audacity, Max MSP, Ableton Live 11 Suite

 

  • Big hurdle in initial recording – the zoom recorder cannot be moved at all or all of it’s setting reset and recording has to be restarted. Very inconvenient but vital for it to function properly.

 

First Recording:  Not much recorded in totality on the boat unfortunately – not the most efficient use of my time. Having to switch back and forth between which microphone to use wasn’t very efficient.

 

Failed max patches: Learning the max plugins for ambisonic decoding was too steep of a learning curve for me to do with the time alotted. I think that will still be a future project of having audio the listener can spatially pan themselves, but the project instead turned into figuring out how to capture essence instead of exactly replicating things.

 

 

Went out a second time and got much better results. Recorded using two programs simultaneously(Audacity and Ableton).

Learned -what metadata is – in order to have the files be read by by the zoom ambisonics software – the metadata has to be edited to decode the file. Many conversion issues between audacity, ableton and this other software.

 

settled on 5.1 – stereo mixdown

Conclusion: I have successfully recorded a lot of the atmosphere of the boat, but capturing the most accurate spatialization has been a pain. There’s still a lot of work to be done with leveling the audio, but I did what I could. There was a push and pull of me wanting to accurately represent the audio and making something that feels engaging for the listener.

 

 

Boat Audio Capture/ Device Making

My friend has a motorboat that he has had a strong connection with for most of his life. I wanted to help him say goodbye to the boat by making a piece that will be able to take him to the boat whenever he misses it after he sells the boat.

 

My plan is to make a multichannel (likely 8 channel audio capture) of the boat using a combination of recording techniques. This includes binaural microphone recording as well as using the ambisonic recorder. Small background – binaural audio replicates a particular person’s pov of hearing and binaural captures a 360 degree audio field that is completely unbiased which sounds quite different but I think are equally important in recreating the sound in a space. My idea is to amplify the recordings by panning them together in various ways spatially to create an effective soundscape. The documentation of this will be through comparing the outcomes of mixing down the multichannel sound.

 

I will be comparing the effectiveness of these techniques

  1. Stereo Mixing
  2. Binaural recording of the space
  3. Max MSP – recreating the space with a patch that recreates the space itself through a sort of room panner
    1. Similar to panning tools that are specifically for multichannel, but this one is for the audience
      1. Example

 

How? – Boat trip – Recording out on a boat with both microphones while we have a small intimate chat about the boat and the history with it.

 

A Spy Microphone ?!

 

There are a couple of people I’ve seen on youtube who have made their own laser microphones which I find to be very compelling. The first video uses a photosensor to detect the vibrations from a mock up window(glass pane) while the one below it uses part of this method but changes it to send different kinds of sound to a speaker. Very cool and opens up a lot of possibilities! DIY-ing this seems really fun and could allow me to do different kinds of audio analysis or storytelling in a piece.

I’m really interested in the history of making these devices(war) and how they can be/ are used in more musical ways now.

 

The last video is describing radios and how interference in them works – not really a project necessarily but still really fascinating. I had a lot of attachment to my own radio as a kid so it’s really nice seeing how they work.

 

 

 

 

Hearing the things we can’t hear: Lightbulb Edition

I’m really interested in the phoneme around us that we can’t perceive as humans.  I was thinking a lot about how sound can create light, but the reverse isn’t possible. Ultrasonic frequencies are the bands of frequency that are beyond human hearing usually above 20-22khz. The frequencies captured in this project go between 0-196khz. I recorded various kinds of lightbulbs in many different places, with the ultrasonic microphone. Initially, I wanted to create a system that can capture and process the sounds of the lightbulb live, but due to technical limitations, I had to scale back the idea. I had to process each sample by hand; and focused instead on the automation of cataloguing the sounds using ffmpeg to create the spectrograms and final video of the images. If I had time, I would combine this with the audio samples to give a better display of the data, but it was too difficult as this was my first time working in bash.

 

Processing of audio – Filtering out audio beyond 20khz with a highpass filter and tuning down the audio by four octaves.

Google Folder of Files

.

The invisible audio spectrum

I’ve recently been really interested in microphones and how they are meant to amplify very specific things. For example, contact mics are meant for capturing the sound of something interacting with the mic. It doesn’t have the best sound quality and is inexpensive because it is supposed to capture generality.

 Although ultrasonic microphones have some of the highest sample rates of most microphones, they aren’t necessarily the best microphones for music production or for radio broadcasters as an example. Depending on the purpose of the microphone, they are biased towards various portions of the frequency spectrum against others, and I find the technology that allows this to happen to be very compelling.

My idea is to mic a long list of things, and create a library of sounds that are constantly being recorded, but aren’t heard in the average human frequency spectrum. I plan to record sounds, places, people, and everyday interactions at the highest sample rate the ultrasonic microphone In order to avoid aliasing, there will be an initial hard bandpass filter that filters out everything beyond 192khz. After, I will filter out the audible information from 0 to 20khz. Then I will tune everything up into 192khz frequencies down several octaves until all of the information that was inaudible is audible and the initially audible portion of the frequency spectrum is no longer audible.

Reading 01 – Photography

I’ve had a very limited interest in photography up until this reading that explores the evolution of photography. The various vantage points that limitations scientists had with different forms of photography is what I think artistically makes them interesting. The paragraph that details how photography doesn’t necessarily display the inherent properties of the object behind an image is something I hadn’t considered and is something that I’d like to explore further. A part of my disinterest (I’m sure)with photography has simply been that I haven’t acknowledged how much of a privilege it is to have the ability to capture any moment anytime from my phone.  I appreciate the reading for showing that it was a long road to getting here.

Looking Outwards #02

Camille Norment – Dia Chelsea

https://youtu.be/1kczbmYVG_E?si=-2rwW2mj1J70DRI4

Camille is utilizing the concept of psychoacoustics to build a site specific music installation. She found the resonant frequencies of the room and used them pitch the bell tones heard in the room. In the video she goes over the historical significance of the bell and why it appears in this work. There’s also a lot of other things happening, but I think this piece is really fantastic and effective in building on a space that can’t be replicated anywhere else. It’s kind of capturing a life in the room that wouldn’t otherwise exist or claiming the space for whatever period the piece was up.