Hello!!
Welcome to my final project post – it’s so sad that the class is coming to an end.
Since I experimented with two projects for the final output, I’ll be walking through Identity Collections (the original aim, capturing faces) first.
Inspiration
After creating two projects focused on surveillance where I was merely the observer putting myself in these situations to capture the invasive, I wanted to push myself and the piece I was making. I wanted to incorporate projection and interaction to make the viewer directly engage with the work, where they are forced to think about the implications of their own face being used or kept within a collection, and how they feel about that idea.
The questions I really wanted to investigate/answer were:
How would people react if their face or identity was captured in a way that blurred the boundaries of consent?
Would the invasive nature of this act stay with them afterward, influencing their thoughts or feelings?
Would they reflect on the ethical implications of image capture and its invasive presence in everyday life?
Brainstorming Concept
The concept drastically changed throughout the semester.
I’ve always had this idea of manipulating people’s faces in real time that would manipulate their mouth to speak. This is technically my idea for senior capstone/studio, but I wanted to see if something could get up and running in the class to capture faces in real time. After lots of really valuable conversations with Golan and Nica, we came to the following conclusions:
1. There is a strong possibility that it just wont work: I won’t get the reaction that I’m looking for, and I may experience a lot of difficulties that I didn’t ask for both backlash from the viewer and/or technical difficulties.
2. It is possible to make it work, reverse projection would be key, but it is beyond my current technical limit especially with the time we have left. I didn’t know Touch Designer well before we started off.
So we changed it, and after talking with Golan, he had an interesting idea about collecting faces in a terrarium like way:
I loved this idea of collection, moving it the faces to one place to another, so to test my technical skills while also capturing faces, the essence of the capture. I set off to capture the faces dynamically into a grid, and felt the terranium level would be the next level of the project if I could get there.
Concept
I’m severely interested in using the viewer as the entity, to further their interaction and engagement directly into the performance. I aimed to use facial capture software using Touch Designer in real time to create a visual experience that allowed viewers to directly interact with the captured collection of faces, extracted from viewers that came before them. I wanted to incorporate projection and interaction to make the viewer directly engage with the work, where they are forced to think about the implications of their own face being used or kept within a collection, and how they feel about that idea.
The questions I really wanted to investigate/answer were:
How would people react if their face or identity was captured in a way that blurred the boundaries of consent?
Would the invasive nature of this act stay with them afterward, influencing their thoughts or feelings?
Would they reflect on the ethical implications of image capture and its invasive presence in everyday life?
Capture System
I played around with what the rig would look like – shoutout Nica for saving the wooden box, and thank you Kelly for donating the wooden box to my cause (along with the wood glue, thank you <3 )
STEP 1 – Visit the rig (Wooden Box)
The idea with this step was the initial step that forces the viewer to choose: do you want to interact or not, and if you do, there’s a slight price that comes with it. Meaning on the outside of the box where the face is, I could write out the words “putting your face in this hole means you give consent to have your face captured and kept” (in need of more eloquent wording, but you get the picture). This was an element Nica & I talked about often, and I loved the subtlety of the words outside of the box.
Recent Find: In my senior studio, I actually had a live stream feed into the piece I was working on and I wrote “say Hi to the Camera”. Most people were completely fine with looking into the camera to see themselves in the piece, however 2 or 3 out of the 45 that came to visit, were adamant about NOT having themselves on screen. Once they were assured it wasn’t actually recording, they looked at the camera and viewed the piece for themselves
TLDR: I think most people would stick their head in?…
STEP 2 – Touch Designer does it’s thing!
more in-depth info in challenges
STEP 3 – Projected Face Grid – Dynamically updates!
Project Inspiration
Tony Oursler, Face to Face, 2012
Michael Naimark, Taking Head, 1979
Challenges
Now here’s where I’ll talk about Touch Designer in depth.
The piece needed to complete the following:
-
- Capture a video in real time using an HDMI Webcam!
- Capture the video on cue, I decided to use a Toggle Button, meaning I am manually tapping record
- Video uploads in unique suffix to a folder on my desktop (person1, person2, etc)
- Touch Designer could read ALL the files in order on my desktop AND could read the most updated file
- Grid dynamically changes regarding how many videos there are!!!!!!! (rows/cols change)
- Most recent video is read and viewed in the Table COMP
- Video within the grids play on a loop and don’t stop even when the grid changes dynamically
Now 1-5 ✅ were completed as seen in the video! But 6-7📍were insanely aggravating.
The biggest problem was that I should have gone to the PC earlier when I first saw the challenge in taking the recording using the Mac.
TLDR: TD was recording in Apple ProRes, but it wasn’t reading the recording when I needed to upload it back into the grid.
When I first encountered the first challenge of recording on Touch Designer and uploading the video to the desktop, I talked with M (thank you M), and we changed the recording of the video from H.264 to Apple ProRes (the only video that could be recorded using a Mac). So at this point, we got the *recording* of the video to work and upload to the desktop. I thought we’d be fine!
Nope!
I had the TD code working, and used Select DAT to read the most recent file in the Facial_Capture folder, and would use Movie In CHOP to read the video and upload it into the Table COMP. But Touch Designer writes that it was “Warning: Failed to Open File” when using Apple Pro Res.
Turns out, Touch Designer needs to be able to encode and decode the recording and there are some idk license complications with Apple, which makes it SO difficult.
https://forum.derivative.ca/t/td-wont-read-quicktime-apple-pro-res-422/172458
SO I tried to record with NotchLC & Hap/HapQ, but even though they download to desktop, the files with the recordings in them DON’T OPEN OR READ!!!!!! They also just won’t be read in TD.
This brought the question of could I export the file manually to an MP4 or a better file version and then send it back in to TD. Nope. This also somewhat defeats the purpose of real time.
Then, I found out that the newest version of Touch Designer SHOULD be able to encode/decode using Apple Pro Res, but then I realized I am on the newest version of Touch Designer. The following article seems like it needs Apple Silicon to work? I don’t feel like that’d be such a big impact, but it just may be because the article states it works with the Apple Silicon chip.
“On macOS, both the Movie File In TOP and Movie File Out TOP have the additional benefit of supporting hardware decoding (depending on the Mac’s hardware). This means that Mac users will be able to take full advantage of Apple Silicon’s media engine for encoding and decoding.3” https://interactiveimmersive.io/blog/outputs/apple-prores-codec-in-touchdesigner/
Anywho, these are just the glimmers of the tech difficulties, but the most solvable reason is that I just needed to test and play with a PC. I was so focused on getting the code to work before I switched over to a PC, and looking back I think I was looking at the steps a little to methodically, and should have played/experimented with the PC from the beginning. Now I need a PC and will borrow one from Bob for the entirety of next semester so I won’t have this problem again. Even though it was a pain, I am pretty confident about my ways around Touch Designer now which is quite exciting that I achieved part of the goal on this assignment.
Outcome & Next Steps
As for the design/materials for the rest of the rig, I have everything:
-
- Kelly’s box , black felt, paint, sharpie (rig)
- transparent mask, the frosted spray paint (Reverse projection)
- mini projector, webcam, tripod stands, extension cords (electronics)
I’m excited to play around with this next semester and see what can come about, since I’m hoping to present something similar at Senior Studio. I’m pretty confident I can get it to work knowing what I know now. Hopefully I can get more technical and explore more interactive works.
Now let’s move on to what was actually presented…
Printed Impressions.
I wanted to make sure I showed something at the final exhibition, and went back to edit my second project.
See Project Comments in-depth: https://courses.ideate.cmu.edu/60-461/f2024/11/11/person-in-time-printed-impressions/1875/
Research Question
How does the sequence of the printing process vary between individuals, and to what extent are students aware of the subtle surveillance from a nearby individual when completing their routine actions?
What event did I choose to capture? Why this subject?
I chose to capture Carnegie Mellon students who are printing out assignments, surveilling their behaviors and habits in this mundane yet quick routine. After a lot of trial and error, I realized the simplicity of observing individuals in this sequence of entering the print station, scanning their card, waiting, foot-taping, and either taking the time to organize p or leaving their account open—creates an impression of each person over time and reveals subtle aspects of their attention to their surroundings, details, and visual signals of stress over time.
I chose this particular subject because not many people would be aware of subtle surveillance when doing such a mundane task, I thought it could also probe more of a reaction, but to my surprise, most people were unaware of my intentions.
What were the project updates?
The biggest takeaway I heard from the critiques were to “extract more of the absurdities,” meaning how could I zero in on what made the people interesting as they printed. Did they fidget? How did they move? How did they interact with the printer? Were they anxious? Calm? Were they quick/fast? Were they curious as to why I was in the space? I really loved the concepts simplicity on something so mundane and was interested to see just how different people were.
Here were some comments that stood out to me:
“Key question: do people print their documents in different ways? Is there something unique about the way people do such a banal thing? It’s been said that “the way you do anything is the way you do everything”. Did you find this to be true during these observations? ”
“I’m glad that you found such a wonderfully banal context in which to make discoveries about people. (I’m serious!) Be sure to reference things like Kyle McDonald’s “exhausting a crowd”, the Georges Perec “An Attempt at Exhausting a Place in Paris”. ”
“I agree that it is hard, like Golan said, to extract information and the interesting bits out of the videos themselves. Maybe what Kim did using voice overs with the Schenley project or subtitles, stalking(?) observations, anthropological notes? But I wonder what more can come out of the project and how can discussions of surveillance and privacy begin from what’s been captured. ”
“I admire you for being a dedicated observer, but I wonder if the privacy issues of this project have been discussed.”
“The collection of people is so interesting because I think I know like 2-3 of them. I think your presence still effects the results and how these people act.”
Video Reasoning
When I went to revise, I realized I had to make them more focused on the act themselves, so I revised the videos to see the similarities within each piece. Did they print the same, similar fidgeting, similar actions of looking at the paper, similarities in looking back to see what I was doing there. I reorganized the videos, but I was actually having a lot of trouble with how to organize them, or if I was doing the project injustice by stitching them all together in this way. My biggest question was how could I show just how powerful these little moments are, how can I help guide the viewer to see what they need to focus on. That thought translated into finding the similarities. I thought about the narration, but was unsure how to write in this format of videos stitched together that I thought it’d be a bit confusing, which is why I didn’t take that route.
Outcome & Next Steps
After talking with Golan, we agreed that the piece wasn’t strong just as a visual video and actually fell flat because the viewers just couldn’t understand what was happening. They needed clear guidance which translated into the need for an audio narration or captions that narrated what was going on. He also explained that it was perfectly fine to focus on each video instead of stiching the moments all together, literally creating a typography of videos accompanied with the narrating captions/audio. I really like this collected idea, and want to focus on it for my next revision. I think the stitching together in this way takes away from the magic of the creep cam and the work. In order to truly focus on the piece, I need to emphasize each video and person.