Contextual Living Room Table

Summary:

Generating interactive 3D contents on a living room table using AR and a haptic puck.

Description:

The living room tables have rich stories behind it – conversation with family, studying for a homework, eating dinner, kids playing with LEGO, and so on. What if these tables in a living room could provide us interactive and engaging contents such as weather forecasting and entertainment for kids? This project allows the user to engage with the AR information and artifacts that are coupled to a physical desk through a haptic interface that consists of an array of small vibrators and a heat pad. The goal is to design text-less and immersive AR interaction techniques that enrich everyday living room table experience.

Process:

1. Test BLE on Unity and iOS
2. Test IR tracking using a webcam (or depth camera)
3. Battery check (drive 4 vibrators and BLE board)
4. Get all tracking, wireless, and actuation working
5. AR contents programming
6. 3D print a package for haptics
7. Prepare a table and a webcam installation

Important tools:

1. BLE board
2. heating pad
3. IR emitter
4. LiPo battery (x5 for backups)
5. webcam
6. webcam setup tools

Deliverables for show:

Space for a table, PC, and a webcam installed on top of table

Media:

Demo video

The first stage of experience is the selection of a content. This experience uniquely occurs on a living room table, so it triggers when the table is recognized through a mobile device. There are 2 categories of content – a weather forecasting and an interactive animation for kids. In weather forecasting mode, imagine the animated 3D sun and cloud appearing on the table and the user feels the vibration of rain or the warmness through the haptic device. This allows text-less and intuitive form of transmitting information. The second scenario is an interactive animation for kids which the user can interact with a simulated dinosaur or a character on the table and being able to interact with it. As the user approaches the device closer to the character, it makes larger vibrations of a walking dinosaur, and when it is very close, the character starts acting toward the device. This would be a perfect interaction to play Pokemon on a table.

A common disadvantage of tangible interaction is that the behavior of interface is limited to its physical form/characteristics and is usually not generally applicable. This solution tackles this problem by turning any passive objects into a control input device using a camera recognition and a 3D projection mapping so that the user can take advantage of unique tangibility of each object. It would be interesting to have a massive collection of tangible interactivities based on the object image database, and the user is capable of customizing tangible interaction at scale.

Assign6-Code

I used NASA’s open data portal and took the meteorite dataset as input. Each row is the history of meteorite landings with latitude/longitude information. Every time the user clicks, it reads the new row and visualizes on Google Maps where it landed. It calls Google Maps API to map latitude/longitude to a city and creates an object for each city. If data with the previously appeared city is observed, it increments the size of the circle. Since there are cases where the city is not included in the API response, I used try/catch to ignore unidentifiable data.

Code-Assign5

When I think of human assistance, a topic that comes to my mind relates to music or rhythm since I’m poor at singing and playing instruments. In this project, I explored an approach to extract sound information from an instrument and to provide real-time visual feedback to the player so that he is always making an ideal sound.

Process:

Using the knock sensing interface from Assignment2, I built an assistive interface for drummers. A piezo sensor attached to a drum captures the strength of hit which is sent to P5JS to visualize whether or not the strength is within an appropriate range. The red line indicates the ideal strength and the gray bar is the actual hit value. The red bar continuously changes as the music progress.

Proof-of-Concept Demo:

Code – Assign4

http://redbearlab.com/blendmicro/

https://www.seeedstudio.com/Blend-Micro-an-Arduino-Development-Board-with-BLE-p-1834.html

Assign3 code