This project is a technical experiment where I attempted to use a polarization camera to separate the reflection on a glass window pane from the light coming from the inside so I can present a 2-channel video of the inside and the outside space side by side.
This is a tool-driven project. I was given a very cool camera and thought it would be fun to make something with it (also a good challenge.) It is a polarization camera, which captures the 4 polarizations of light simultaneously with a special sensor that has polarization filters of 0, 45, 90, and 135 degrees on its Bayer filter rather than red pass, green pass, and blues pass filters designed to make a colored image.
I was initially inspired by how polarizers are used to eliminate reflections and glares with water and glass in photography. I thought it would be interesting to capture the duality of windows — that it lets light through while reflecting it at the same time, that it lets us see ourselves and others at the same time, and that it separates (by physically dividing) and unifies (by combining two images on to one surface) two spaces at the same time. By capturing one video clip of one surface and separating it into two channels to be presented side by side, the contrast between these two different yet intermingling images investigates the simultaneity and unity of different moments and spaces.
My workflow:
- Find an interesting window.
- Set up the camera in front of it in a way that my own reflection cannot be seen.
- Record a ~10s clip.
- In post-processing, choose 2 channels that capture the outside and the inside with the best separation (usually 0 and 90 degs.)
- Enhance the difference between the two frames and adjust them so that they look as similar as possible visually while having the largest achievable separation between the outside and the inside.
- Present them as a 2-channel video.
Further work:
This project focused more on the experimentation of tools and processes rather than the soundness of concepts. It is quite weak as far as typology and content go. I have a few ideas in mind that could make the content more interesting. When I’m in a big city like NYC, I’m always fascinated by the stark contrast between the inside and the outside — the pristine inside of corporate lobbies and the sticky street where the homeless take shelter separated only by a sheet of glass.
A stock photograph to illustrate what I mean:
https://www.alamy.com/stock-photo/homeless-man-sleeping-in-shop-window.html
As for the capturing process, I think 2 regular cameras with polarizing filters might work better than the FLIR Blackfly. The black fly does not allow for precise adjustments of the filter angle on the field so it’s hard to gauge the success until I get into cv2. Having two regular cameras would also allow me to capture color.
The main challenge I encountered was the balance of light. If either the inside or the reflection was too bright at any spots, it overpowers the other and wipes out any information from the other side. Essentially, unless the light is very balanced across the frame, at any given pixel, I can only capture one of the sides. One idea I had to combat that was to match the FPS with the grid’s AC frequency. LED lights usually flicker at 120HZ with a 60HZ grid and dip to less than 10% peak luminosity at their dimmest points. So if I can record at 240fps or higher when the ambient light is low (blue hour or later), I can easily pick out all the darkest frames (which would have most of the reflection) and the brightest frames (which would have the most of the inside.) This can also be used in combination with polarization.
I looked through IEEE explore and ACM library but found very little literature on this topic, which was very surprising since glare reduction via polarization filters seems to be a very useful thing to do in the industry. I’m sure there are better algorithms to utilize the data I have than the stuff I came up with. If anyone knows anything about this please let me know:)
Additional information:
The Degree of polarization map (L) and Angle of polarization map (R) I calculated from the 4 channels using this repo: https://github.com/elerac/polanalyser Didn’t find any good uses for this information.
BTS:
