yanwen-ARSculpture

(^ It’s actually 50 secs long idk why it shows up as over 2 mins??)

Timestamp

I decided to create a work to (sort of) recollect my experience from 2017 to 2020 when I was staring at my postcard holder that I used to collect all sorts of ephemera and pieces of printed matter over the last few years. I couldn’t recall the exact reason why I started to collect these things but I ended up with this postcard holder being all filled up (and even it weighs a lot, it was one of the first things that I decided to carry along with me when I moved from New York to Pittsburgh this Summer).

Forgot to take a photo before dumping all the pieces out : (

I was thinking about how this habit of collecting ephemera tracked the passing time and what I could reflect upon them, so I created this set of four AR sculptures using images from printed tickets I collected as tracking image sites. Each sculpture represents one of the four years and is made up of two parts: a 3D model of a roll of film using photos I’ve taken in that year as texture, covered by an orb with some text I wrote down when I was looking through the photos.

AR sculpture example

Besides learning about how to do augmented image and image tracking, figuring out how to show transparent background video in Unity was also a great learning experience (although it took a while…). I first recorded my writing in Procreate and then edit out the background with chroma key in Premiere, and finally imported into Unity as video texture with alpha channel. I also embedded sounds in the videos. The sounds are extracted from other videos I took in that year.

I kept the movement of the sculpture pretty minimal and focus on the composition of the scene, including the tickets that I used as trackers. Although the tickets have more specific dates other than the years (and they are actually all from MoMA), I focused on the years only.

Tickets I used
Tickets I used

As for why MoMA tickets… I actually don’t have a specific reason other than the quantity (ended up having 14 like why) and the quality of the images they used are good for being trackers.

 

axol-ARSculpture

Chalk Shark

As I kid I drew on the sidewalk a lot, and indulge myself in my fantasy world. Thus for this project, I want to make such a virtual world come alive to me through AR.

I wanted to make a 3D chalk shark drawing, that surfaces from the ground drawing. However, I’ve been testing in my apartment hallway and wasn’t testing outside, and it really didn’t look as good as I wanted it to be 🙁

For the programming portion, I modified the code of the AR Controller so that the mesh is always placed sideways (90 degrees) towards the camera.

I made the 3D model and handpainted the texture(in Substance painter and Photoshop)  to give it a hand-drawn quality. I planned to hand particle systems and flipbook textures to animate the water, but I didn’t get that to work, so I’m just animating the two pieces of mesh that represent the water splashes by bobbing them up and down.

Hallway Test Vid:

More Documentation:

 

gregariosa-ARsculpture

[My clips are taking hours to load… the official video will come soon.]

I’ve made a fictional story where a woman goes to a therapist and talks about how she sees hallucinations of “Nipples of Venus” everywhere she goes. I’m hoping that the hallucinations are a metaphor for the emotional baggage that people have trouble explaining through words.

 

lampsauce-ARSculpture

Augmented Creatures

I ended up using Blender for the entirety of this assignment instead of Unity. It is not really AR in that it is not realtime. Instead, I used image tracking within Blender to map certain image targets. Then, I rendered the animation I wanted superimposed over the original video. I definitely think using Unity probably would have been easier to do than what I did. However, I still learned a lot about FBX files (the main reason I did use Unity is because I could not properly make an FBX of the creature asset I made).

Process

Creating the creature: I made one generic creature in Blender using a series of pentagonal toruses. I animated them as I wanted and the style is inspired by my Creature Clock for deliverable03.

 

Image tracking demo

Tracking the video: I prerecorded this rather mundane scene of my family’s dining table (honestly, we never eat here tho). Using the Movie Clip Editor within Blender, I detected the features and tracked them throughout the clip. Then, I assigned a plane and a scene using the trackers so that I could map my assets onto that plane.

AR scene arrangement

Arranging the scene: I then created multiple copies of the original asset. Here, you can see the camera movement is calculated based on the previously mentioned tracking.

Rendered animation

Time to render: Because I wanted the shadows of the objects to be realistic, I used the cycles renderer. Unfortunately, these measly 372 frames took close to 12 hours to render. Each image here is actually a PNG (ie. it has a clear background)

Composited animation

Putting the two parts together: This was honestly a tad disappointing because after 12 hours of waiting, seeing that black border was not a fun time. I still think it looks cool but, no need to worry because cropping is a thing.

Cropped final render

Cropping: After the final cropping, there is a bit of compression, so the quality is a bit reduced, but I am super satisfied with how the shadows appear on the table and the wall.

sweetcorn-arSculpture

Proposal for the Removal and Burial of “Turner’s Made Fast”

A project I made for Turner’s is still installed in the basement of Doherty Hall. I mourn the loss of the relationship, but cannot seem to healthily move on from it while this monument to our disordered affection still stands.

Pictured above is a plan in Unity for the grave of the sculpture, which should be imposed in AR over the small amount of grass in my backyard here in St. Louis, over 600 miles away from Pittsburgh. It will be put into action once I receive a reply to the following email:

To whomever it may concern,

[Image of Sculpture]

I’ve made a grave mistake regarding the installation of this sculpture in Doherty Hall B-Level right outside of B302. It has been installed for almost a year and I wish for it to be promptly removed and shipped to the following address:

[My Address in St. Louis]

Please speak to me if funding is an issue.

Thank you,

Benford Krummenacher

I will bury the sculpture once it arrives, like a family pet. Maybe then I can finally move on to healthier relationships.

 

I was previously working with videos, but found them distracting and not tied to the concept of planning/blueprinting.

junebug-ARSculpture

AR Meditation

Follow this guided meditation to help you relax. Base your breathing patterns on the movements of the entity.

After watching this video and following the meditation, are you relaxed? Or are you more anxious as you watch this boding entity hover over you for an extended period of time?

For this project, I decided to base my project on stress and anxiety. During childhood, whenever I dealt with stress and anxiety, I loved going to my backyard and just stare up at the sky. I enjoyed the soundscape that it immersed me in – the sounds of my pool and fountain, trees rustling, and birds chirping always seemed to calm me down. Since I was back in my childhood home and living in a stressed time, I’ve been going back out there to take Zoom and school breaks.

Then, I started playing with this idea of a conflict between audio and visual. After building my sculpture and running some Unity tests, I realized that since it’s all digital, the installation is much easier to manipulate than if it was in physical, real, space. I thought what if I made the sculpture slightly foreboding as it hovers over the viewer/audience? Then the audio would be trying to get the audience to meditate and de-stress, but there is a lingering feeling of unease from watching the video and watching this thing bob up and down slowly for almost 30 minutes.

My initial idea for this project was to get this sculpture to be floating above my pool because I thought playing around with reflections and transparency would have been cool, but I had troubles with getting the sculpture to appear directly on top of the water, so I had to switch gears.

tale – ARSculpture

This project is an AR portal to the top of a cliff that the user can use to explore a virtual mountain area and find a hidden item.

Screenshots:

gifs:

Inspirations:

I used to explore the apartment complex with my friends when I was little. With great creativity, we imagined different worlds and put ourselves into the shoes of various people and living creatures. One of the scenarios was a treasure hunter, a common recreational game that asks for the teamwork among players to obtain hints and find the hidden items.

Inspired by it, I visualized the portal to another world that hid one of the keys (the item the players have to find and collect). Instead of making it vertical like a door to another world, I located the gate on the floor. This was inspired by my another childhood curiosity and creativity regarding manholes. I always wondered what’s at the end of the manhole, as I’ve heard of different usages of manholes like “it’s for sewage” or “it’s a path to access buried cables,” but have never been inside one. As a child, I would imagine the manhole to be something more grand than its actual usage, like serving as a secret transportational tunnel for spies or a portal to another weird interesting world like Wonderland.

Reflecting these childhood imaginations, I narrated the video as if I’m part of a team of two and am following the hints to collect all three keys before other players do.

Process:

I modified free 3D models I downloaded from online in Blender to create the virtual world beyond the portal. Then, I imported those to unity and scripted accordingly using the AR template.

Because it was my first time using Unity to create my own AR project, I spent a large number of hours looking up how to obtain certain effects I wanted. Although I occasionally got annoyed at C# scripting for it not doing what I asked for (at one point, all I was looking for was a constant rotation at a constant speed, which should be able to obtain with a simple line or two of code, but I had no luck and had to satisfy with other alternative effect…), I think situations like that would less frequently happen as I work on more projects on Unity and get use to C# language syntax and the Unity program itself.

Texturing also took a good amount of time, as one of the models refused to show the texture in Unity. Thanks to Hochi, I was able to learn and apply UV mapping and unwrapping in Blender, which I found it very interesting. It also helped me understand why computer graphics course is requiring students to take 3D calculus and matrices before taking it.

Reflection:

Overall, I enjoyed learning Unity and working on this very first AR project, and I’m happy with the outcome of the project. I appreciate everyone in this class for being supportive and providing opportunities to observe and try new things.

Big Snake

This project is meant to channel the very relatable moment when you wake up one day to find that a giant snake has broken into your house (or rather through your house) and is refusing to leave. Why is it here? What does it want? Will your insurance pay to fix the whole in your wall?

The following video contains a lot of swearing: