Author: BosiBox

“Little Bosi”: Being Seen

Introduction

In this interactive digital art project, I explore the emotional and energetic effects of being observed versus being alone. Using Unity as the primary platform, I’ve created a digital “me”—Little Bosi—who resides within a 3D living space on a computer screen. Little Bosi reacts differently based on the presence or absence of an audience, offering a poignant reflection on isolation and human connection, inspired by my own experiences during the pandemic and its aftermath.

Concept

This project delves into the transformative power of attention. During the pandemic, I endured long periods of solitude, disconnected from people and outside signals. Weeks would pass without meaningful interactions, and sometimes I would go days without speaking a word. It felt as though I lived inside a box, cut off from the external world. During that time, my mental state was one of exhaustion and sadness.

The process of emerging from that state highlighted how every interaction and moment of attention from others created ripples in my internal world. A simple gaze or fleeting connection could shift my emotional energy. This concept inspired the idea for Little Bosi: an embodiment of these emotional dynamics and a visual representation of how being seen impacts the human spirit.

Interaction Mechanics

When Alone:
Little Bosi enters an emotional down state, expressing sadness, boredom, and exhaustion. The digital character performs actions such as slouching, sighing, and moving lethargically. The world around Little Bosi gradually fades to a monochromatic tone, symbolizing emotional depletion.

When Observed:
When someone approaches the screen, Little Bosi transitions to an interactive state, showing joy and energy. Actions include smiling, waving, and sitting upright. The environment regains its vibrancy and color.

Techniques

  • 3D Scanning: I used Polycam to scan 3D models of myself (Little Bosi) and my living room to create a digital space resembling my real-life environment.
  • Animation Development: The animation library was built using online motion assets, which were refined through IK rigging and manual keyframe adjustments. Transition animations were crafted to ensure smooth movement between emotional states.

 

  •  Placeholders: For the current concept and testing phase, video placeholders are used to represent animations while final transitions are being completed in Unity’s Animator.
  • Interactive Coding: Unity’s OpenCV plugin powers the interaction system. Using the plugin’s object detection and face recognition models, the camera identifies the presence and position of individuals in front of the screen. This data triggers Little Bosi’s behavioral state:
    • Face Detected: Activates interactive state.
    • No Face Detected: Switches to solitude mode.

Reflection

The project aims to create a “living space” that bridges the digital and emotional realms. By using 3D modeling, animations, and environmental changes, I evoke the contrasting experiences of loneliness and connection. The audience becomes an integral part of the narrative; their presence or absence directly influences the digital world and Little Bosi’s emotional state. Through this work, I hope to resonate with audiences who may have faced similar feelings of loneliness, reminding them of the importance of connection and the subtle ways we leave traces in each other’s worlds.

Next Steps

  1. Finalizing animation transitions in Unity’s Animator to seamlessly connect Little Bosi’s emotional states.
  2. Enhancing the environmental effects to amplify the visual storytelling.
  3. Expanding the interaction mechanics, possibly incorporating more complex gaze dynamics or multi-user scenarios.

 

People In Space

Touch Designer & Polycam, Point cloud interact

To use Polycam to scan the environment into a point cloud model and import it into TouchDesigner. Then, using a computer webcam, capture the movements of a person in front of the camera. Combine Feedback, Time Machine, and Noise in TouchDesigner to create particle-based trailing effects related to human motion, such as flowing lines. The captured motion data will influence the imported point cloud model dynamically, creating an artistic interaction.

Steps

  1. Environment Point Cloud Import:
    • Use Polycam to scan the environment and export the point cloud model.
    • Load the scanned point cloud model into TouchDesigner.
  2. Human Motion Capture:
    • Use a webcam to capture the motion of a person in front of the camera.
    • Extract keyframe data of the movement for further processing.
  3. Feedback, Time Machine, and Noise Integration:
    • Use these tools to create particle trailing effects, such as flowing lines that respond to the motion.
  4. Point Cloud Model and Motion Interaction:
    • Map the captured human motion data to the point cloud model in TouchDesigner.
    • Dynamically influence the point cloud to create artistic effects based on human motion.

Hands Conversation

The Hand as an Extension of Thought

In this project, I explore the hand as a sensory antenna, capturing unconscious, subtle movements that reveal layers of thought and perception. The hand, often viewed as a tool for action or expression, is, in my perspective, an extension of the mind itself—a medium through which our thoughts and emotions materialize, engaging us tangibly with the world. Here, I examine the idea that cognition isn’t isolated in the brain alone; hands are equally integral to the thinking process.

HANDS CONVERSATION

Concept and Inspiration

The foundation of this project lies in capturing the unconscious use of the hand—those involuntary movements that occur without deliberate intent yet often go unnoticed. From nail-biting, rubbing hands, twisting fingers, clenching fists, tapping fingers on a table, to touching the face or rubbing the arm, these gestures reflect the hand’s connection to our internal world. They embody silent conversations, carrying nuances of tension, contemplation, and even comfort.

Methodology 

I approached this project with the intention of remaining an unobtrusive observer. My subjects are teachers and students in classroom settings across four distinct environments. Filming without prior notification, I capture natural scenes where each person, fully engaged in conversation or thought, unconsciously expresses themselves through hand gestures. By ensuring that my subjects remain unaware of the camera, I preserve the authenticity of each gesture, allowing their actions to unfold organically.

To further emphasize the hand movements and protect individual privacy, I use deliberate audio manipulation in post-processing. The tonal characteristics of voices are intentionally blurred, with layered audio tracks that obscure specific conversational content. This abstraction creates an auditory backdrop that distances viewers from spoken language, directing attention solely to the hands and the silent, yet expressive, language they convey. The combined visual and auditory effect encourages viewers to engage with these gestures as independent expressions of thought.

Reflection 

Throughout this project, I’ve found that hands—whether fidgeting, clasped in stillness, or making rhythmic taps—reveal an unspoken dialogue. Each movement, seemingly minor, becomes a window into inner states and subconscious expressions. By focusing on this, the project reimagines hands not as simple tools but as sensory extensions of thought, constantly communicating even when words are absent.

LookingOutwards on AR APP: Artivive and Pokémon GO

Artivive

Artivive is an augmented reality (AR) platform focused on the art field, allowing artists to transform their static works into interactive augmented reality experiences. With Artivive, artists can overlay videos, animations, and other dynamic elements onto their physical works. Viewers simply use the Artivive mobile app to scan the artwork and can see the corresponding augmented reality effects on their screens. However, the application only works with Artivive extended artworks.

https://www.artivive.com/

Pokémon GO

This is a game that incorporates AR technology, allowing players to use their phone’s camera to “capture” virtual Pokémon in the real world. The game combines location-based services and AR technology, creating a highly interactive experience.

https://pokemongolive.com/?hl=en

Conversation With Hands-WIP

Concept

The hand is the extension of thought. In this project, I will continue to capture the movements of the hand. The hand is the sensory antenna through which we perceive the world. It is not merely a tool for executing tasks or achieving goals; it is an extension of our thoughts and intentions. Through the movement of the hand, thought is materialized and transformed into tangible interactions with the world. Thinking does not happen solely in the brain—hand and body movements are also part of the cognitive process.

In this project, I aim to capture the unconscious use of the hand, focusing on the small, involuntary movements that occur without deliberate intention. These subtle actions, which often go unnoticed, reflect the hand’s role as an extension of thought and perception.

Biting nails, rubbing hands, twisting fingers, clenching fists, tapping fingers on the table, touching the face, stroking the chin, rubbing the nose, caressing the wrist, or rubbing the arm…

What am not 100 percent sure
  1. CAMERA: How to capture? Where to set the camera? Hidden Camera?
  2. A designed SCENE? Or capture “wild” scene?
  3. How much tech is involved? Hand tracing? Or edit it by myself?
  4. How to present?  Video Collage?

First Direction: A set scene, Invite others, 30 min 1 to 1 chat (topic unsure)

Second Direction: Take my camera to public space.

Encounters in Motion

1. Marina Abramović’s  Work
The Life , 2019

 The Life — a 19-minute performance realised through volumetric capture and presented in Mixed Reality — was the first large-scale public exhibition of anything in this new medium.

I find it interesting to use augmented reality technology to turn the very presence of an artist into a collectible artwork. However, I think this piece mainly relies on the artist’s fame and the novelty of applying new technology to art as its selling points. Personally, I prefer her 2010 work The Artist Is Present. This piece uses silent eye contact to convey the tension between people, which reminds me of the word “gravity.” It is a dialogue and fusion between the aura of the artist and the viewer.

The Artist Is Present, 2010

 

2. Dolly Zoom

The dolly zoom creates the illusion of a changing distance between two objects through a combination of zooming and adjusting the camera’s distance. This technique effectively conveys a character’s inner tension by shaping an irrational perception of spatial distance. I’ve also seen this technique used in old sci-fi films without CGI to create a scene like “shrinking a spaceship down to a nano-scale.” I have a personal connection with this technique. In a TRS course project during my freshman year, I accidentally discovered and intentionally used this technique to create a project about the perception of distance. At the time, I had never heard of the dolly zoom and didn’t know what it was. It was only later, when I learned about the work Serene Velocity, that I realized I had a similar approach to that piece, though I was 50 years late.

 

3. The Encounter of Two High-Speed Trains

I absolutely, absolutely, absolutely love this work. It is a piece that captures the chance encounter of two high-speed trains running parallel to each other purely by coincidence. I highly, highly, highly recommend others to watch it. I believe it is one of the best examples of what this project represents.

The two trains run parallel, overlapping repeatedly; in the darkness, every brightly lit window frames a diverse array of human figures. It is a grand collective memory, where many small individuals live their vivid lives in this era of Chinese high-speed rail. In a sense, it resembles life itself: traveling alongside many others, sometimes in sync, sometimes surpassing them, sometimes drifting apart, and ultimately, we part ways.

High-speed rail is a significant symbol of this era in China. During the 2024 Spring Festival travel rush—a short period of massive population movement as people return to their hometowns—the national rail system is expected to transport 480 million passengers. High-speed rail has become a symbol connecting “development” and “home.”

Whenever I’m on a train and experience an encounter like the one in the video, I always wave vigorously to the other train. Sometimes, there is someone waving back; more often, no one notices at all, but I don’t mind. Because I understand that people meet by chance, then part ways, never to see each other again. It is one of the most ordinary things in life. If you have ever encountered someone waving to you when the trains crossed, that fleeting moment might have been the only intersection in your entire lives.

(Here is the original video link. The original video is on Bilibili, which is kind of like YouTube in China, with 4.5 million views. The version on YouTube is a re-upload by someone else, and the video quality is very low.)

https://www.bilibili.com/video/BV1CF4m1F7k7/?buvid=YD49F0D731D7E609496FBE9F5690060109E1&from_spmid=main.my-history.0.0&is_story_h5=false&mid=QXc5VhI5grddeADYL6TUNw%3D%3D&p=1&plat_id=114&share_from=ugc&share_medium=iphone&share_plat=ios&share_session_id=9CD00E30-B9B3-42D8-89E1-A8BE00611AD7&share_source=GENERIC&share_tag=s_i&timestamp=1728487696&unique_k=X87DvgM&up_id=2330984&vd_source=175b6495a033376bc68000c6706abcdd

 

 

Sunday Ingredients

Practice part 1: Ingredients

In this project titled Sunday Ingredients, I photographed the objects I touched with my hands throughout the day on and arranged them sequentially based on the time of interaction, creating a list of objects. These items constitute my day on that Sunday—The Ingredients of 9.29.2024.

I believe that daily life is made up of countless ordinary, trivial matters, and each of these small events is triggered by an object. For instance, drinking water as an activity might be initiated by a bottle of mineral water, or perhaps by a combination of a cup and a faucet. Therefore, the mineral water bottle itself, or the combination of a cup and a faucet, can imply an action—in this case, drinking water. When we document a series of objects, we can imagine and deduce a sequence of behavioral trajectories. For example, if the table shows remnants such as a candy wrapper, a book, a pair of glasses, and a mouse, we might visualize that the owner of these objects was perhaps sketching while eating candy at some point. However, we cannot definitively state that the owner removed their glasses, read a book, ate a candy, and then played with the mouse this morning. This is because these objects form an overlapping trace of different moments, each leaving behind its own mark.

Desk Composition

The hand is the agent of intentional choice. It serves as the bridge connecting my mind to the outside world. If I were an ant, the hand would be my antenna; if I were an elephant, it would be my trunk. The act of touching an object with the hand represents the embodiment of will. When an object is actively touched by the hand, it reflects the need of that particular moment. Thus, when the touched objects are linked together in chronological order, they record my life. If I touch the faucet, refrigerator door handle, milk, cup, and bread in succession, these objects collectively suggest the event of breakfast.

Breakfast Sequential Items

In this project, I documented the main objects my hands actively touched on Sunday, 9.29, starting from when I woke up until I lay in bed preparing to sleep. I sorted them chronologically and divided them into eight time segments.

The Ingredients OF 9.29.2024.

Practice part 2: Collage

After collecting these objects that reflect my activities throughout the day, I wanted to use these “ingredients” to create a collage that represents my day. I took a photo of my hand from that day and used all the objects I interacted with as materials to collage an image of my hand. This collage, titled Hand of 9/29/2024, symbolizes my day.

HANDS OF 9.29.2024

 

I took 65 photos of the objects I touched that day, cropped each one into 5×5 small squares, and obtained a total of 1,675 material images.

65 Ingredient Item’s Photoes

Library Resource

For the target image, I converted it into a grayscale image and divided it into 15×20 small squares. Then, I compared each target square with the material squares in the library one by one and selected the closest match to replace the original square.

In my comparison function, I used OpenCV’s Canny Edge Detection, ORB (Oriented FAST and Rotated BRIEF), and SSIM (Structural Similarity Index Measure) to perform a comprehensive similarity analysis between the target image and images from the library.

        1. Convert the target image and library images into edge maps using Canny edge detection.
        2. Use ORB to compare the similarity between the edge maps of the target image and the library images.
        3. If the ORB similarity score is greater than the set threshold value (e.g., 0.5), then proceed to evaluate further.
        4. Scoring mechanism: Calculate the SSIM value between the grayscale version of the original target image and the grayscale library image. Then, combine the SSIM score with the ORB score using a weighted formula. The weights are yet to be determined and represented as variables for now.

Collage Camera

Collage Camera Workflow:

  1. Capture a static initial image and apply stylization:
    • Use a camera to capture a static image as the original material.
    • Apply stylization to the image, enhancing the contrast between light and dark areas, simplifying details, and creating more abstract, block-like color areas. This step aims to highlight the outer contours and reduce complexity in the image, making it easier for subsequent contour extraction.

2. Block segmentation and line information analysis：

  • Segment the stylized image into multiple blocks, with each block containing part of the image.
  • Analyze and process the line information within each segmented block. This may involve edge detection, shape recognition, or further geometric feature analysis within the blocks. The results of the analysis guide how the image is reassembled or used for subsequent collage creation.

3. Prepare a collage material library:

  • The images in the collage material library should contain minimal information to simplify the composition.

4. Stylize the images in the collage material library:

  • Apply stylization to the images in the library, extracting their primary outer contours.

5. Match line features:

  • Compare the line features of each block from the target image with those of the images in the collage material library, selecting the image with the most similar line features to replace the block.

6. Collage assembly:

  • Assemble the collage by pasting the selected replacement images into the corresponding blocks of the target image.

The Artistic Expression of Photography

Photography was initially seen as a mechanical and objective tool for scientific observation, fulfilling the need for accurate documentation. However, subjective choices like lighting, composition, and exposure reveal photography’s artistic potential. I believe this interplay between subjectivity and objectivity provides the possibility for photography to become an art form, as photographers construct images through their own aesthetic and philosophical frameworks, thereby creating a secondary interpretation of reality. This process also reflects the distinction between “artistic recording” and “artistic creation,” demonstrating how photographers balance objective documentation with subjective expression. An example is electron microscopy, designed for objective scientific recording, but often resulting in visually stunning images of microscopic structures with fascinating textures and symmetry. These choices, such as magnifying specific parts and adding color, not only serve scientific purposes but also elevate the images into artistic expression.

Pocket Postulating: Lightpainting, Panorama, Slowmo

Lightpainting: Usually we use lightpainting to capture the path of light. Long exposure creates mesmerizing light trails, making it great for artistic representations of movement. When the object is still, lightpainting allows more details to be captured. When the object is moving, lightpainting will record the trace. The following are lightpainting pictures for 1 second exposure and 3 second exposure. I move my head while the exposure time.

Panorama: This technique usually is for capturing the full breadth of a scene or subject, like wide landscapes, cityscapes, or long horizontal objects. Since the panorama photo is a collage of series continuous captures of an object from different points or angles, we can use panorama technique to create photos with different perspective of a same object. The following are panorama pictures of a painting in Andy Warhol Museum. The first’s capture path is parallel moving with the object. The second’s capture path is changing the angle i stand.

Slowmo: Usually we use slowmo to record Fast-moving objects to reveal details that are invisible to the naked eye. Here I use it to record the process of “tried yawn-to-true yawn”. I want to see how our facial muscles work differently.

https://drive.google.com/file/d/1N8TLes2dLv5W0qVgBzE_6vEWV1ocmdJk/view?usp=sharing