TLDR: I took a break and had the robot supervise my break time: a self-imposed timeout.

urscript code: https://cmu.box.com/s/vjpo6s4ji6vdo2vp1iefm85jxokj3x56

Subject: myself or any singular person.

Topic of focus: what happens when people are forced to be bored? how does the presence of a robot arm affect how people act?

Inspiration: shooter, candid shot, this project about robot surveillance, MITES@CMU

Observation of myself as a result: I fidget a lot. And I think as time went on, I got sleepy and I cared less and less about the camera (I didn’t realize this is how it looks when I nod off in class).

This project is a bit last minute, due to two main components for my original idea not working. One thing I kinda got working was URScript (thank you golan!) but to be quite honest, even that wasn’t working very well. I think there’s some math involved that makes looping in a half circle (in a very simple, maybe dumb manner) that doesn’t work due to division by zeros due to zero radius (idk what this even means). Even after debugging my script, the math didn’t work out and I ended up resorting to just making scripted motion just by moving the robot to the position I thought it needed to be at.

The original idea was to use the robot arm to make animated 3D objects, which proved to be way too large in scope for this to be completed. I spent the majority of the last 2-3 weeks working on developing a script to automate the creation of multiple Gaussian splats, a new-ish way to create 3D objects. This didn’t work (it’s still not working). Thus, speed project was enacted and this was made. I really struggled with URScript — it took me 1.5 hrs to debug why my script wasn’t compiling (apparently if statements also need to be ended with an ‘end’ in URScript). But even after getting to compile, there was something about the computed trajectory that resulted in the robot just not being happy with me.

some thoughts: I think this is just a little sloppy. If I were to redo it, I would make a good filming box for myself. There’s too much background clutter that was captured due to the angles the robot arm was filming from. I think idea-wise, this project still explores the concepts of surveillance and how we occupy space/exist differently with weird pieces of tech added to our environments. I will be continuing to explore these ideas, but I think as an entirely different project. This robot arm frustrates me.

setup for filming:

debugging process for original idea and modified idea: 

Printed Impressions.

Research Question

How does the sequence of the printing process vary between individuals, and to what extent are students aware of the subtle surveillance from a nearby individual when completing their routine actions?

What event did I choose to capture? Why this subject?

I chose to capture Carnegie Mellon students who are printing out assignments, surveilling their behaviors and habits in this mundane yet quick routine. After a lot of trial and error, I realized the simplicity of observing individuals in this sequence of entering the print station, scanning their card, waiting, foot-taping, and either taking the time to organize p or leaving their account open—creates an impression of each person over time and reveals subtle aspects of their attention to their surroundings, details, and visual signals of stress over time.

I chose this particular subject because not many people would be aware of subtle surveillance when doing such a mundane task, I thought it could also probe more of a reaction, but to my surprise, most people were unaware of my intentions.

 

Inspirations

1. Dries Depoorter, The Follower (2023-2024)

https://driesdepoorter.be/thefollower/

I found this absolutely incredible project (for my looking outwards 4!) by an artist who takes an individual’s Instagram picture and uses a surveillance camera company called EarthCam to show the real-time process of that individual attempting to capture the very image they posted to Instagram. Depoorter essentially exposes the real-time workflow behind an Instagram capture, illustrating the the ironic relationship between personal photo consent and public surveillance consent

B’s Typology on Surprise, capturing people’s surprise reactions as they opened a box.

and… Special shout out to Hysterical Literature & Paul Shen’s Two Girls One Cup

Initial Project Focus & Redirection 🙂

I was initially really obsessed with subtle surveillance that could capture a series of fear or shock committed by an unknowing individual and even capture some information about them that isn’t revealed verbally.

However, much conversation and redirection from Golan and Nica was exceptionally helpful to uncover what exactly is a quiddity, and how I should capture a person over time. I was really grateful for my talk with Nica because they really reminded me the importance of trial and error, to quite literally go out and figure out what works well (and doesn’t) with the creep cam. Golan’s reminder that sometimes the more simple project is often the most powerful was really helpful.

 I did a LOT of overthinking since I felt like it was too close to the first project, but I’m confident that although I’m using the same capture method, it’s intention relates etc a completely different capture, one that simply observes 🙂

So with their wonderful advice, I got to work!

Capture System – Creep Cam

1. 1. Discuss the system you developed to capture this event. Describe your workflow or pipeline in detail, including diagrams if necessary. What led you to develop or select this specific process? What were your inspirations in creating this system?

I built this capture method for my previous class project where I would bump into people who were on their phones, and would then ask “what are you doing on your phone”. I wanted to stay with turning the Creep Cam into capturing something more positive, but still staying in line with subtle surveillance as the purpose of the project is to incorporate subtly film the entire footage without asking for permission.

What is a Creep Cam, you may ask? It’s a small 45-degree prism used by predators on the subway to subtly look up people’s skirts. So, why not repurpose this unfortunately very clever idea in a completely new way?

Ethics was heavily discussed, and since the footage is in a public place, turns out it’s all good–no permission required . After discussing my concerns with Golan, he suggested the PERFECT solution: a Creep Cam.

Close Ups of the Rig!

 

Set Up – Creep Cam in the Wild

When employing the creep cam to capture the event, I need to wear black clothing. On the day of I wore all black: dark pants, top, coat – the real deal to make sure the camera was as unnoticeable as possible! (Pictures don’t show the outfit I wore the day of)

I chose my location in the UC Web Printer, an alcove that’s nice and small to make sure my creep cam could see all the little details.

 

 

Printed Impressions

I was pretty shocked to see the results to say the least… I was able to record 12 people!!! My personal favorites are 5, 9, 11, 13, 14!

Insights & Findings

1. 1. Advantages of the Dual Camera 

A big comment that was given on my previous project was that I should learn to take advantage of the creep cam’s dual camera. Since the creep cam has a dual camera of both the top and bottom half of a person, it perfectly captured the stress you could feel from them having to wait for a printed image. Many of them would tap their fingers on the table, tap their foot, or even sigh! I was really excited because the environment perfectly captured each individual’s process at an interesting angle.

2. Most Individuals did NOT realize I was filming them

Most people who went to use the printer would ask if I was printing something out, when I would say no, they would rapidly go to print out their stuff. After reviewing the footage, I think some people were somewhat aware, but wasn’t completely reflecting the possibilities of what was going on.

3. Person in Time is Clear!!!
I was most excited with the fact that it perfectly captured a quiddity, a subject of a person completing an action in time, and you could witness the varying factors of stress or habits among individuals. Printing is such a mundane task that requires someone to wait for the output especially in such a digital world, that it really boils down to focusing on the individual’s focus at the task at hand.

4. Happy Accidents

I was extremely happy with the added aspect of surveillance towards revealing personal information, since most people using the printer failed to log out of their account. I was able to use this to catch a glimpse into the possibilities of who they are and what they’re working on!

 

 

now for the project 🙂

 

Person 1 – Liam C. – First Year, Dietrich College

 

 

 

Person 2 – Unknown

 

Person 3 – Unknown

 

Person 4 – Unknown

 

 

Person 5 – Unknown

Person 6 – Tomas R. – Sophomore, Business

 

Person 7 – Unknown

Person 8 – Unknown (Again)

Person 9 – Nick B,  Junior, Business

 

Person 8 – Amy C,  First Year, CFA

 

Person 11 – Kobe Z, 5th Year, Electrical & Computing Engineering

 

Person 12 – Nikolaj H, Masters, Mechanical Enginerering

 

 

Person 14 – Unknown

 

 

 

Evaluation of Printed Impressions

 

The Hand as an Extension of Thought

In this project, I explore the hand as a sensory antenna, capturing unconscious, subtle movements that reveal layers of thought and perception. The hand, often viewed as a tool for action or expression, is, in my perspective, an extension of the mind itself—a medium through which our thoughts and emotions materialize, engaging us tangibly with the world. Here, I examine the idea that cognition isn’t isolated in the brain alone; hands are equally integral to the thinking process.

HANDS CONVERSATION

Concept and Inspiration

The foundation of this project lies in capturing the unconscious use of the hand—those involuntary movements that occur without deliberate intent yet often go unnoticed. From nail-biting, rubbing hands, twisting fingers, clenching fists, tapping fingers on a table, to touching the face or rubbing the arm, these gestures reflect the hand’s connection to our internal world. They embody silent conversations, carrying nuances of tension, contemplation, and even comfort.

Methodology 

I approached this project with the intention of remaining an unobtrusive observer. My subjects are teachers and students in classroom settings across four distinct environments. Filming without prior notification, I capture natural scenes where each person, fully engaged in conversation or thought, unconsciously expresses themselves through hand gestures. By ensuring that my subjects remain unaware of the camera, I preserve the authenticity of each gesture, allowing their actions to unfold organically.

To further emphasize the hand movements and protect individual privacy, I use deliberate audio manipulation in post-processing. The tonal characteristics of voices are intentionally blurred, with layered audio tracks that obscure specific conversational content. This abstraction creates an auditory backdrop that distances viewers from spoken language, directing attention solely to the hands and the silent, yet expressive, language they convey. The combined visual and auditory effect encourages viewers to engage with these gestures as independent expressions of thought.

Reflection 

Throughout this project, I’ve found that hands—whether fidgeting, clasped in stillness, or making rhythmic taps—reveal an unspoken dialogue. Each movement, seemingly minor, becomes a window into inner states and subconscious expressions. By focusing on this, the project reimagines hands not as simple tools but as sensory extensions of thought, constantly communicating even when words are absent.

I created a pipeline that takes a video and creates an abstract representation of its motion.

see it for yourself : https://yonmaor1.github.io/quiddity

code : https://github.com/yonmaor1/quiddity

videos:

WHY:

I’m interested in analyzing how people move, specifically how people move together. Musicians were an obvious candidate, because their movement is guided by an ‘input’ that the audience is able to hear as well. I was curious about abstracting away the musicians motion into shapes that we can follow with out eyes while also listening to the music, in order to see what in the abstraction is expected and what isn’t. Certain movements correspond directly to the input in ways that are expected (the violist plays a forte and moves quickly so their motion blob becomes large), while others are less expected (the violinist has a rest so they readjust their chair, creating a large blob in an area with little movement otherwise).

how:

video -> optical flow -> Gaussian Mixture Model -> Convex Hull -> b-spline curve -> frame convolution

in detail:

1. I used a video of the Beethoven String Quartet Op.59 No.1 “Razumovsky”

Credit: Matthew Vera and Michael Rau – violins. David Mason – viola Marza Wilks – Cello

2. Optical Flow

I used OpenCV in python to compute dense optical flow of the video. That is, I represent the video as a 2d grid of points and compute a vector stemming from each of those point, representing the direction and magnitude that that point is moving in in the video.

3. Gaussian Mixture Model

I filter out all vectors below a certain threshold, then use the remaining vectors (represented in 4 dimensions as x, y, magnitude, angle) as data points in a Gaussian Mixture Model clustering. This creates clusters of points which I can then transform into geometric blobs.

aside: why GMM? GMM clustering allows for both non-spherical clusters and overlapping clusters, both of which are unachievable with other popular clustering algorithms like k-means or nearest neighbors. My application requires a lot of irregularly shaped, overlapping clusters as the musicians overlap one another while they play.

4. Convex Hull

I now take each collection of points that step 3. yeilded, and compute its convex hull – that it, a closed shape tracing the perimeter of the cluster.

5. B-spline curve

I can now smooth out this convex hull by treating it as a set of control points of a b-spline.

6. frame convolution

This step simply smooths out the output video, in order to create a less sporadic final video. This essentially transforms each frame into the average of the frames surrounding it.