https://experiments.withgoogle.com/imaginary-soundscape

I found this project and thought it was a really cool use of ML with a lot of potentials. In the demo, the model generates sound based on the street view, giving the user a full audio visual experience from the purely visual street map data. I thought it really utilized DL to its strength — finding patterns for massive datasets. With only a reasonably sized training set, it can be generalized to the massive street view data we have. The underlying model (if I understood/guessed correctly) is a CNN (which is frozen) that takes in the image to generate an embedding, and another CNN that is trained but discarded at inference which takes in the sound file to generate an embedding as close to the one generated by the image CNN for the paired image. The embedding for each sound file is then saved and used as a key to retrieve sound file at inference time. With more recent developments in massive language models (transformers,) we’ve seen evidence (Jukebox, Visual transformers, etc.) showing that they are more or less task agnostic. Since the author mentioned that the model sometimes lacks a full semantic understanding of the context but simply match sound files to the objects seen in the image, these multi-modal models are a promising way to further improve this model to account for more complex contexts. It might also open up some possibilities of remixing the sound files instead of simply retrieving existing sound data to further improve the user experience. This might also see some exciting uses in the gaming/simulation industry. Windows’ new flight simulator is already taking advantage of DL to generate a 3D model (mainly buildings) of the entire earth from satellite imagery, it’s only reasonable to assume that some day we’ll need audio generation to go along with the 3D asset generation for the technology to be used in more games.

Nine Eyes of Google Street View by Jon Rafman

This work is an archival and conceptual project started in 2008 by artist Jon Rafman. These images are all screenshots from Google Street View’s image database, back when it was a new initiative without the kinds of regulations that it has nowadays. At that time it was (and still somewhat is) “A massive, undiscerning machine for image-making whose purpose [was] to simply capture everything, Street View takes photographs without apparent concern for ethics or aesthetics, from a supposedly neutral point of view.” The work meditates on the implications of this kind of automation of image-making, and at this kind of scale.

I am so glad I came across this project. It really struck me going through all of these images, I just could not stop looking. As a collection for photography’s sake, they work together to really poetically capture subtle moments of life, whether it be ugly, mundane, funny or really beautiful. But as you look through these photos, it feels really eerie too. It feels like you shouldn’t be looking at a lot of these (I mean.. because we shouldn’t) and yet here they are, moments of life captured with no regard for its subject. Some of these photos made me audibly gasp, like the one of an inmate running or one showcasing what looks like a kidnapping. And then in the same collection you have a guy mooning the camera. Or a set of white laundry billowing in the sun. It’s just strange and so eerie. I really, really love it though.

I think this also holds up so well in our current times with the development of computer vision and artificial intelligence. The technology is certainly developing and yes, the ethical side is too – but definitely not enough considering many of the issues posed in Rafman’s work are still relevant today.

Here is the image collection:

https://conifer.rhizome.org/despens/9-eyes/20180103101851/http://9-eyes.com/

Domestic Tension by Wafaa Bilal, 2007

This is a networked durational performance. The artist confined himself to a gallery space and broadcasted himself to the internet. The audience can view, chat with him, and is also given the power to control the pinball gun to shit him. Through this work, Bilal intended to highlight the violence and racism of US culture after 9/11.

This work is different from many other net arts in that it’s using ‘network’ to portray violence, disconnect, contradiction and the BAD. This work reminds me a lot about Marina Abramovic’s Rhythm 0, 1974. In both works, the artists themselves serve as the medium/net to provoke audience interaction and reveal the underlying BAD. 

How Do You See the Disappeared? A Warm Database (Mariam Ghani, Chitra Ganesh, 2004)

Link  | Video

This project was developed in response to the U.S. government detaining immigrants, most of whom were Muslim, following 9/11; many others were also subject to questioning. The “warm data” in the project’s title stands in opposition to the data generated from such questioning; it is “deeply personal but non-identifying”, as opposed to common forms of data categorization such as place of origin or birthday. The project is currently inaccessible, but is separated into 4 parts; I liked the 3rd and 4th parts, which was the questionnaire that collected the “warm data” and the interface for viewing such responses. I thought this project was really interesting because of how it utilized databases and data that focused on humanizing and turned its focus away from categorization, and how it can evoke an emotional response from the viewer despite the fact that there is nothing known about the person who submitted the response.

IMAGE OBJECTS by ARTIE VIERKANT

Image Objects responds to the shift in art-making practices and the definition of the work of art, where contemporary art began focusing on its online representation. It is composed of sculptural works placed in between the physical objects and the digital image in order to place emphasis on the fluid boundary between them. The digital images were created by printing image-based works on aluminum composite panels so that they would appear three-dimensional. Then, the altered digital documentation of them was circulated online, which means that each documentation is a unique work of its own.

At first, I was drawn to this artwork because of its appealing visual qualities with simple yet vibrant colors. Yet, as I got to know more about the idea behind it, I was intrigued by its focus on the online circulation of the artwork. As my artwork in the future could also be digitally circulated, it led me to consider the interactions of my artwork through the audience’s lens (ex. phone camera).

https://brianhouse.net/works/joyride/

I found the project Joyride, created by Brian House, especially interesting and also humorous. The ability for the artist to be able to determine the path of a stolen phone emphasizes the dystopian implications behind pieces of advanced technology: with a smartphone at hand, anyone’s location is trackable. At the same time, the reenactment of the thief’s journey is humorous in that it seems to mock the thief in that even though they have stolen the phone, everything is still under control and surveillance. The fact that the project was done through google map street view furthers the profound control and web of information on the internet.

Natural Process

Link

Natural Process is an installation that translates the Google search engine homepage to a painting, and then takes the painting back to digital media through webcam. The painting was purchased by Google.

I find it really interesting because it blurred the boundary between digital media and physical media. It also makes the Google homepage unfunctional, forcing the audiences to notice the design of the page, which is usually overlooked.