Reflections

Created by Yasha Jain, Michael James, and Kabir Mantha

The piece

“Reflections” explores power relationships through the manipulation of angle and time. A rotatable double-sided reflective surface (commonly known as a whiteboard), inherently forces spectators into two disparate experiences. It is impossible for one person to see both sides at once. While someone tilts the whiteboard, they can only see the affect on their own experience. They are thus naturally inclined to disregard the experience of “the other” (side).

The piece uses two projectors, two cameras, a whiteboard, accelerometer, and signal processing to create this interaction opportunity. There are projectors directed at each side of the whiteboard, positioned so that a person may stand in front of the board but not block the light. Two cameras are mounted on top of the whiteboard, pointing in opposite directions. An iPhone is affixed to the top of the whiteboard, so that the y-axis of its accelerometer corresponds to the tilt of the board. The camera feeds and accelerometer data are processed by Max/MSP/Jitter to create images mapped with Millumin. In Max, an operator can switch between different recipes that dictate how inputs mix into outputs.

One recipe is using the tilt of the board to control the playback of recorded video. This recipe starts with a recording period where people can move around in front of the cameras. Then in the playback period, the tilt controls the location in time of the “playhead”, allowing the interactor to move forward and backward through the recorded frames one by one or rapidly.

A second recipe layers live video with delayed video to create a double image. The tilt of the whiteboard controls how delayed the second image is.

Getting here

Before our first workday, we talked about various ideas such as projecting on a mirror ball, but nothing really stuck. When we all first got together to work, we quickly landed on the idea of projecting on two sides of a whiteboard. The way in which only one side of the whiteboard could be seen at once led to a discussion about privilege and the physical relationship between interactors and the angle of the board. While we first discussed the use of found footage, we settled on live camera because it lends itself to a more generative and performative foundation.

After playing with the basic set-up, we realized we could do powerful things if we had data about the tilt of the whiteboard. Because our smartphones are packed full of sensors, we quickly settled on using an iPhone’s accelerometer data to interpret the angle of the board. With our set-up and this tilt data in hand, we brainstormed different ways of mixing things together. One of the most exciting ideas we landed on was using the tilt of the board to move backwards and forwards through recorded video.

At this point, we started building a Max patch and stepped up our set-up by including the iPhone and directional lighting. We experimented with these variables over two different sessions, observing about a halfway through the way projected light bounced off the whiteboard and created murky figures on the architectural surfaces of the space. We quickly jumped on this observation, finding it to be one of the major elements of the piece.

At this point, we added a false wall so that both sides of the installation could experience this reflection. We also tweaked our recipes, creating one where a current frame and a past frame are composited to create a double image. We got all the pieces (laptop, projectors, cameras, Millumin, Syphon, and Max) up and running and shared the work with our class.