A BodyPix-driven tool for reflecting about the COVID-19 pandemic through performing gestures captured as sillhouettes.

Tools used: BodyPix, OpenCV, LowDB, Glitch.com
Live demo: https://glitch.com/~bodypix-covid-grief 

Some context

We are experiencing a period of profound collective grief. In a recent  interview with Harvard Business Review, David Kessler, expert on grief and co-author (with Elisabeth Kübler-Ross) of on Grief and Grieving: Finding the Meaning of Grief through the Five Stages of Loss, says that we are feeling a sense of grief during the COVID-19 pandemic:

“…we’re feeling a number of different griefs. We feel the world has changed, and it has. We know this is temporary, but it doesn’t feel that way, and we realize things will be different…This is hitting us and we’re grieving. Collectively. We are not used to this kind of collective grief in the air.”

Grief hurts. One way we can process, heal, and/or cope in this moment of grief is by listening to our own hurt, and listening to the hurt of others in solidarity.

healing happens when a place of trauma or pain is given full attention, really listened to.

— Adrienne Maree Brown

The most profound way to process grief with others is being physically present with them. This is because, according to scholars and musicians Stephen Neely (professor of Eurythmics at CMU) and Émile Jaques-Dalcroze, our bodies are considered the first instrument; we come to know our world through the immediate tangible interactions with our environment.

“before any experience is understood in the mind, it has to first resound through and be felt in the first experiencing instrument, the body.”

— Jaques-Dalcroze

But grieving with others becomes profoundly hard when we must be apart, when being close to loved ones means we might make them sick.

It’s hard to carve out a moment to take stock of how we’re feeling, and even more difficult to share feeling with others beyond text, video, and our own limited networks. In the words of artist Jenny Odell, art can serve as a kind of “attentional prosthesis.”  For example, work such as Nina Katchadourian’s Sorted Books invite us to dig into our own libraries to create our own sculptural book phrases.

Taking a leaf from Nina Katchadourian’s book, I’m hoping to create a lens for observation of our felt experience during the pandemic, one that opens up space for participation and solidarity.

How might I create a capture system for expressing grief as a display of solidarity?

The general workflow

The subject: a body in motion

I chose to abstract the body in motion into an animated silhouette. It not only protects identity of participants, but amplifies our sense of solidarity with them. We can see ourselves in more abstract bodies. They also invite play.

 

The capture system

What tools like Ml5.js and others offer us is a way to make tools, or lenses of observation, more widely accessible to non-experts. Platforms like Glitch made deployment friction-less.

The workflow

At the end of the capture, you can download the capture you’ve created, and submission is optional.

The captures

I opened up the tool on a trial run with my cohort, with several captures over 1-2 days. I processed their submissions into a collection of gifs:

We may find that we have lost a lot during this pandemic. 😢
Expressing sadness can be an act of release. How do you feel?

There will be a moment when the pandemic is over. Imagine it 😀  How will you dance?

What piqued my interest

A lens for reflection. I’ve created a tool for easily (and privately) expressing emotion through gesture, particularly in a time when body-centered reflection is needed. The project is less about the gallery of captures, more about the opportunity to explore with the body. Nevertheless, I am most excited about capturing (and remembering) the pandemic feels in a visceral way.

Bodypix. This is a relatively new port to Ml5.js. The browser-based tool makes capturing poetic movement in time more easy to use and accessible.

Glitches & opportunities

Tone setting / framing. I’m used to holding these kinds of reflective conversations in person. How I could set a reflective tone with no control over its context of us was largely a mystery to me. Early feedback seemed to emphasize the tool didn’t provoke them to reflect deeply.

Glitchiness in gifs. Inconsistency across machines produced wildly inconsistent (and glitchy) gifs.  How quickly certain machines can run the bodyPix model and draw frames in seemed to P5.js vary wildly.

Seamless workflow for storage of gifs. My server (temporarily) broke. Lowdb didn’t readily store the created gifs in an image format, so . My server filled up after only ~20 submissions, after which I opted to only store silhouette frames (which I then post-processed into GIFs by manually downloading the frames from Glitch). Ideally, the pipeline from capture to gallery would be fully automated.