Final Project Proposal

3D scanning technology can restore the 3D object better than flat photography, but they are all static. I want to use touch designer for this project because it can achieve a lot of dynamic interactions. I have tried various things, and two main sections gave me some inspiration. I decided to focus on them to do interactive digital character capture.

1. I can successfully scan the point cloud with a mobile app, restore it in touch designer, and after adding background music, the point cloud will change with different tracks

2. I found a pipeline on the Internet that uses teachmachine to control the model in touch designer. And tried to train my own model with teachamchine.

I have not yet successfully connected the first and second parts. I will spend some time to continue researching. I think the possible solutions are:
The effect I want to achieve in the end is that I made digital animations for each classmate’s avatar. Whoever walks to the computer/puts the photo in front of the computer, the avatar will appear and dance (if my technology can do it, maybe it can dance with the recording of the person himself!).

Final Project Proposal

I think I will use the final project as an opportunity to expand on my person in time. There are lots of avenues I can take to explore my project further, some in the video processing part of the project (currently working on a fluid simulation, but there are other possibilities), the image capture (im thinking about using an RGBD camera to have more dimensions to characterize motion by), and the final presentation (I still want to polish the interactive demo I had made for the second project).

Pursuing this for the final project will also give me time to talk with Stephen Neely, something I was planning to do but didn’t find the time during the person in time project.

Final Project Proposal – WIP

Narrative Project

For my final project, I would like to experiment with 3D scanners & depth sensors to create abstract volumetric video footage. I want to take advantage of the distorted scenes created from limited depth perception to convey an abstract narrative/poetry. And if time permits, I would like to try to make the footage audioreactive. (tbd after convo with Nica & Golan)

Inspo:

Faith Kim (2017) —Pittonkatonk festival immersive 3D project 

does it have to be fixed? could it be moved around?

….if this doesn’t work then would like to attempt lyrical typography capture of the narrative/poem from composites of letter scans.

Buggin’ Out

Concept:

Invite people to come view objects collected in Schenley Park (such as leaves, flowers, rocks etc.) through microscopic (displayed on my laptop monitor) and capture their reactions with a high speed camera (and an audio recording of their voice, recorded on my computer). I’m expecting them to see bugs, but I am also interested in having them observe some other things, specifically things they have collected*. In general, I’m anticipating some reaction of surprise, some of wonder, and some of disgust. This certainly happened to me when I was looking at the microscopic images. What was especially interesting to me was that it made the observed objects feel totally new! It makes me realize that observing them with the naked eye can only ever go so far, and that there is always more to see! This kind of discovery feels eye-opening and mind-expanding.

*Note: I have previously invited a bunch of people to go on walks with me to Schenley Park, during which I asked them to pay attention to small objects (among a few other things) and to think about small objects as individual things to be observed, but also as interrelated parts of a larger landscape. I’m including some worksheet images of what kinds of objects people were observing at the end here.

Besides capturing people’s facial reactions with the high speed camera, I plan to take videos with the studio microscope of a collection of small organic materials found in Schenley Park (including plants and non-living materials like rocks or trash). 

Things to think about:

How could I/ should I display this together? I’m wondering what the best presentation format might be. Also, is there something interesting I could do with the high speed camera that I’m not thinking about? 

Bug’s View of Person? (but not if it’s on a leaf on a tree… right?) Or under a microscope..

Lillian suggested that it could be fun to show people in a sort of bug’s eye or worm’s eye view of the people. This sounds potentially fun to me. But not sure how to execute. Any other thoughts or ideas? I want to do something cool, challenging, and fun…



WIP Final Project

For the final project I want to do a spin off from my last project. Essentially, I want to capture these memorable last moments with my college friends. With three of my friends we regularly meet every Saturday 7pm at our friend’s apartment to watch k-dramas together. We each bring snacks to eat while screening whether it is something we cooked or baked or trader joe sweets. There is a tv in the living room that we hooked up the computer to and all four of us sit in a row on the futon. I wanted to capture this core memory in a form that I can remember and be immersed in when I look back at this work years later.

This time instead of just using a go-pro camera I wanted to capture the depth and this works perfectly with the kinect camera because we all face in one direction watching the screen.

I first just set up the work space on the windows computer. I have the kinect SDK downloaded and did some test trials working with its software.

I think another important aspect of this capture is not only the visual but also our conversations because we would be talking about a totally random thing while keeping our eyes locked on the screen. Sometimes we have two different conversation going on at the same time and it gets very chaotic but its the only time that I put down all the load and enjoy the moment.