l.o. 04
m. tayao
lab section E

‘the witness-machine complex’ examines how technological interventions in sound can and do mediate our ability to communicate with one another in tangible spaces. the case study of this project is the Nuremberg trials in 1945-1946, where then-newly developed electronic audio technology was used to enable live translation during court proceedings. the translation machines used red and yellow flashing lights to change the speed or repeat audio that was then fed into the headphones of the prosecutors and witnesses. often, these interventions would add confusion and stagnated time to the trials, pauses that were then cut from recordings of the proceedings.

in this exhibition, computational sound artists abu ham dam collected seven of these moments of confusion and re-enacts them using machines that mimic the technology of the translators used in 1945. he uses these staged moments to highlight how the relationship between testimony and the technologies that confer them disseminates and distorts how truth and justice operates in global contexts.

While looking into sound and computational artwork, I stumbled upon the sonic exhibition ‘Rupture’ by Camille Norman. The work was a performance pieces in which a series of panel glass frames lay broken or shattered in a room with microphones hanging above that emit a vibration. I think this work was makes an interesting comment on the way sound waves and vibrations produce types of factures within the air. In some sense, these can seem random, but in others these waves can be manipulated to produce a certain outcome of experience. The idea of path in this sense is taken to understanding from the microphone to the experiencer’s ears. I think this in a sense relates to the paths of the computational design. Weaving and interweaving within each other to finally reach the eye at one given instant. Amongst the chaos and the parameters of these waves and progressions, there results a single moment in time of experience.

https://www.norment.net/work/objects-installations-ind/rapture/

One computational sound project that really impressed me is the work of Ryoji Ikeda. Ikeda’s installations work with presenting data through a cinematic immersive experience, and reimagining how data can be visualized through multiple sensory avenues. His work (primarily audio as well) works with the concept of binary input, where there are merely two possible outputs of data (e.g on and off, black and white). Together, it synchronizes with binary visuals that create this immersive method of understanding large complex structures like genetics and biology.

What I especially appreciate about this piece is how natural the integration with technology and music was undertaken. It’s always a risk to “overcompute” a creative practice within these types of projects, which undermine the beauty of arts based performances. However, the project reimagines how sound can be portrayed with binary outputs.

I admire this project as the system of connecting the alternating environment with the sound to create an immersive experience for the visitors. By having different sensors and converting the lighting variation and day-night cycle to usable data, it creates a series of changes to the visitors’ visual and acoustic movement. For the algorithms, they first used sensors or detectors to track the changes in lighting variation. Then, the 500 artificial neurons that are created will respond to the data converted by the detectors. This cycle continues so that the real-time data will be analyzed to create an immersive experience. The artificial neurons, they are running on the Attiny 85 with Arduino. In addition, since there are also LED lights inside, and they are placed in a hexagon shape, this kind of placement creates a sense of conflict with the continuously changing visual and acoustic movement. The overall experience will create a world that contains order and chaos. Therefore, I think it aims to create an immersive experience for the visitors.

Link: https://www.creativeapplications.net/environment/floating-codes-the-spatial-topology-of-an-artificial-neural-network/

The piece I picked is Radio Rocks by Dove Bradshaw. These pieces consist of conical sculptures of different types of rocks- Wissahickon schist, Pocono sandstone, and a basalt mixture. These sculptures were created in this shape to emmulate ancient cairns that used to be used as Neolithic astronomical markers and functioned as antennas.

Within these conical sculptures, Bradshaw placed three radios each that received waves from different frequencies. Local, world band short wave, and outer space. These radios received these frequencies and emitted them at low levels. Dove Bradshaw does a lot of work with stone, salt, and things that evoke themes of nature, the world, and space. This work is a vestige of neolithic creations with themes that intersect with the work that she makes.

link to the artist’s website: http://www.dovebradshaw.com/works/Radio%20Rocks.htm

For this project, a work that I found to be very interesting was “Purform – White Box, Audiovisual Performance” from 2010. I was mesmerized by the abstract visualization of sound on the giant screens, the animations smooth and flowing along to the music playing. This is something that I admire because I can recognize how difficult something like this must be to achieve. I personally very much love music, and to see it visualized through waves and other abstract shapes with this technology was very exciting. I’m not quite sure about the algorithms needed, but I would assume that they used trigonometric functions in order to make these kinds of shapes. The artist needed to be able to visualize how they wanted the sound to be portrayed on these screens, because this kind of project could have been interpreted in many different ways. The artists’ vision needed to be specific and they needed to use coding and other technology to make their vision a reality.

Watch the video here.

AIVA is a software that aims to create AI-generated music. It gives its users the chance to select from a preset of tools or import their own inspiration in order to create a unique song or beat of their own. The creators aim for AIVA to be used as a supplement to other projects. Game developers, composers, and more can use AIVA to add an emotional supplement to the creative process. As someone who loves music, this project is especially interesting to me as I feel as if music is very key to the emotional experiences we find in other forms of entertainment. Many of these aspects intersect, creating a very personal experience for the consumer. What is different about AIVA, though, is that this experience can be created by an everyday person who might not be too versed on how to compose such music for themselves. This allows those who wish to experiment, and gives those who are using it for professional purposes to buy the copyright to the music that is created, which is something not usually seen in apps like these.

Clare Katyal

I researched Seiko Mikami’s Desire of Codes. This is a project which uses a variety of sensors and small lights, as well as cameras to follow the movements of visitors. The movements trigger a response throughout all of the units, and lights are pointed on the visitor. I find it very intriguing how Mikami programmed all of these units to work in tandem to create an interactive experience for visitors. The sound created gives a slightly uneasy feeling, as the sounds given off by the guests are translated to sound like insects flying. I think this is an interesting idea, especially because of the creators intention to play with time and space.
https://www.youtube.com/watch?v=XjUC1xjRek0
2011, Desire of Codes, Seiko Mikami

This week I’m looking at a project by Luke DuBois titled Moments of Inertia. This is a series combining the music of violinist Todd Reynolds and “interactive video”.

To create the images seen, DuBois’ built software that reorganizes and maps Reynolds’ audio loops onto visual imagery. The computer uses analysis of changes in pitch, amplitude, and performance style to re-animate footage of people moving and interacting with the world in various ways (playing a sport, using a phone, etc.). This method of perception can be very uncomfortable and fascinating in its own right. I think the contemporary process of music making that gives this work its roots is also very interesting. I wonder if it would have the same effect if more ‘traditional’ music was used instead.

Moments of Inertia (2010) from R. Luke DuBois on Vimeo.

The project I am looking into this week is Ryoji Ikeda’s Test Pattern series. These projects are a series of installation pieces that use sound and black and white visuals to describe the flow of binary data, 1s and 0s. The pieces act as a magnifying glass into the extremely dense stream of data in our world. I find this work really fascinating because of how it immerses the viewer into the data stream. It not only does this visually, but also with sound. The world of technology which surrounds us now uses 1s and 0s to operate; however, we only see this from extremely far away. We are immersed in data everyday but not to the scale that is described in Test Pattern.