People in Time Project Ideas

Here’s a few ideas I’m mulling over for the project. For this project I’m really interested in exploring the unique dynamics and kinematics of human body. Though there are some interesting ideas listed below, I don’t feel that my final framing is contained within.

  1. Walking to a Beat*: A video processing algorithm that generates skeletons for walking people, measures the frequency of the gait and picks a song where the beat matches that frequency. Songs would be presorted by tempo.
    1. https://www.youtube.com/watch?v=6Ron-Ikenfc : films often put music over someone walking or doing some other mundane task to give it this sense of being intense, important, or intentional — this scene from spider man shows what happens when that audio is removed. What would happen if we created a system where that audio was added back in?
    2. https://github.com/aubio/aubio
  2. Movie Path*: An algorithm that processes scenes from movies and uses photogrammetry combined with skeleton tracking to figure out exactly the orientation and path of a camera relative to someone in scene. Use that information to generate a path for the robot arm and film someone to add into the original scene.
    1. Which films?
    2. How contemporary would the films be? — There would definitely be a bias towards contemporary film.
    3. Scale? Films use giant booms to hold and guide cameras – would UR5 robot arm be able to achieve  the necessary motion for this project?
      1. Maybe we could put a squirrel into the scenes?
    4. Similar Idea: Use the algorithm to look at individual movie scenes and encode camera position relative to the floor and the focal plane — sort all scenes this way. Make this searchable by camera position. Use this to find movie scenes that could accept characters from other videos — put squirrels into films? 
  3. On Body Camera Rigs: There are a lot of camera rigs that stabilize the jitter and motion that is inherent in handheld filming, could I create a system that does the opposite? — That moves and responds to human input. The goal here would not be to create a jittery noisy video, rather to create a system that dances and moves and rotates in response to some human input
    1. maybe slow motion camera would be good?
    2. Could a gimbal system be repurposed to do this?
    3. https://www.videomaker.com/buyers-guide/camera-rig-buyers-guide
    4. https://www.adorama.com/cdagcjib.html?gclid=EAIaIQobChMIkfjP2_WB6AIVlozICh36ZgzDEAQYAiABEgJjV_D_BwE&utm_source=adl-gbase
  4. Long Exposure People Pixel Averaging: capture the trails of people by making an algorithm that averages pixels over time with wait the pixels where people have been (a pixel has both position in grid and position in time)
  5. People responsive systems*: set up a set of highly responsive systems, that will rotate in the slightest amount of air in a populated place set up a video camera to capture the rotations and frame the entire scene through these responsive systems
    1. Also: put responsive things into the air that people could move like smoke or micro bubbles or dust
  6. Dust cam: Create a camera set up in front of a light source at just the right angle so that it will pick up the microscopic dust particles in the air
    1. As people walk past the dust particles would get disturbed in a distinct way.
  7. Novels shots:
    1. Building cut away like the hallway scene in oldboy
      1. Creating a very flat side scroller  way of interacting with this
      2. Could an algorithm take like 3 wide angle shots and stitch together a super flat image?
      3. http://www.cs.technion.ac.il/~ron/PAPERS/ieee_mds1.pdf
      4. https://mathematica.stackexchange.com/questions/34264/how-to-remap-a-fisheye-image
      5. https://docs.opencv.org/2.4/modules/imgproc/doc/geometric_transformations.html?highlight=remap
      6. Maybe video photogrammetry (where a set of frames for photogrammetric model is generated at each point in time)
    2. Other shots that might make people look a little funky/be fun:
      1. Fisheye lens
      2. Parallelized imagery with hyperbolic mirrors(see bellow)
      3. schlieren optics
        1. https://en.wikipedia.org/wiki/Schlieren_photography
        2. https://pdfs.semanticscholar.org/3267/9a2ab1d35774a4b859323e5d7548efb45660.pdf
    3. Side Scrolling Everywhere*: (Connected to above — Novel shots idea):
      1. Use a series of cameras and a flattening/warping algorithm to create super flat scene that looks like a side scroller game

Hallway scene from oldboy