Category: Uncategorized

Final Project: Responsive Computer

Problem

“Until now, we have always had to adapt to the limits of technology and conform the way we work with computers to a set of arbitrary conventions and procedures. With NUI(Natural User Interface), computing devices will adapt to our needs and preferences for the first time and humans will begin to use technology in whatever way is most comfortable and natural for us.”

—Bill Gates, co-founder of the multinational technology company Microsoft

I think that gesture control interface could have great potential to help people interact with computing devices naturally because gestures are inherently natural. Gestures are a huge part of communication and they contain a great amount of information, especially the conscious or unconscious intentions of the gesture-doers. They sometimes communicate more, faster, and stronger than other communication methods.

General Solution

In this perspective, I want to design a gesture user interface for a computer. When people are sitting on a chair in front of a computer, their body gesture (including posture or movement) shows their intentions very well. I did some research and I could find a few interesting gestures that people commonly use in front of a computer.

When people are interested in something or when they want to see something more closely, they lean forward to see something in detail. Reversely, when people lean backward on a chair with two hands on their heads staring at somewhere, it is easy to guess that they are contemplating or thinking on something. When they swing a chair repeatedly or shake legs, it means that they are losing interests and become distracted.

The same gestures could have different meanings. For example, leaning forward means the intention to see closer when people are looking at images, but it could mean the intention to see the previous frame again when they are watching a video.

I am going to build a gesture interaction system that could be installed on computers, desks, or chairs to recognize the gestures and movements of a user. According to a person’s gestures and surrounding contexts (what kind of contents he/she is watching, what time is it, etc), the computer will interpret the gestures differently and extract implicit intentions from them. This natural gesture user interface system could leverage user experience(UX) of using computing devices.

I am also considering to add haptic or visual feedback to show whether the computer understood the intention of the gestures of the user as input.

Proof of Concept

The system is composed of two main sensors. The motion sensor is attached under a desk so that it could detect the movements of legs. The ultrasonic sensor is attached to the monitor of a laptop so that it could detect the posture of a user, like an image below.

The lean forward gesture could be interpreted differently based on the contexts and the contents that a user is watching at that time. I conducted research and found correlations;

  1. When a user is seeing an image or reading a document, they lean forward with the intention to look something closer or in detail.
  2. When a user is watching a video, they lean forward with a surprise or an interest, with the intention to look at the recent scene again.
  3. When a user is working on multiple windows, they lean backward for thinking for a while or see the overview of all windows.

Based on this intention and the contexts, the system I designed responds differently. For the first case, it zooms in the screen. For the second case, it rewinds the scene. For the last situation, it shows all windows.

Also, the system could detect the distraction level of a user. The common gestures when people lose interest or become boring are shaking legs or staring at other places for a while. The motion sensor attached below the desk could detect the motion and when the motion keeps being detected more than a certain amount of time, the computer turns on music that could help a user focus.

Video & Codes

(PW: cmu123)

codes

A device to help with a restful night of sleep

PREMISE

The sleep machines currently available are often not very interactive. You set the timer before you fall asleep and hope that you fall asleep before the timer finishes or you leave it on for the entire night. Another reason people tend to leave their sleep machines on the entire night is because they are afraid of loud noises waking them up during the early hours of the day.

PROPOSAL 

An interactive sleep machine which helps you fall asleep and then stay asleep! As the user falls asleep, soothing sounds will be played. Once, the user has fallen asleep the sound will be turned off. During the night, if external noises reach above a certain dB level, white noise will begin to play to cancel out the noise.

PROOF OF CONCEPT

I used a pulse sensor, speaker and a microphone. The pulse sensor was used to identify what part of the sleep cycle the user is in.  The microphone monitors the sound in the room to understand whether the speaker should be triggered or not. In the mode to fall asleep, the speaker plays a calm tune and once the pulse drops it moves into sleep maintenance.

 

 

Continue reading “A device to help with a restful night of sleep”

Crit #3 – Multiple Timers

Image result for kitchen timer

Problem: When in the kitchen and cooking a big meal (say, Thanksgiving dinner), I often have multiple timers going.  Between the microwave, my phone, my roommate’s Google assistant, etc., they can be hard to manage, especially when many timers are physically locked to their positions on the appliances.  This can be an issue for not only those with low mobility, but because each timer is often on a different type of interface (touch screen, keypad, twist timer), it can affect those with low dexterity as well.  Timers should be manageable and adaptable to user needs.

Solution: I want to solve the problem in two ways: by combining the timers into one place, and making input methods modular such that users can select the input that works for them e.g. lever, button pad, knob, etc.  The user should be able to easily discern which of their set timers is going off even though they are all now co-located, though, and this is done by unique audio cues for each.  They should also be able to know which timer they have shut off, and which are still going, based on sound.

Proof of Concept: My proof of concept is a system of 3 timers with one on/off button, one knob to set time, and one knob to select a timer.  Each of the timers can be set and turned off independently.  When one timer is going off, it adds to a melody that plays all of the currently on timers’ contributions.  Each timer has a distinct sound.  Users can also turn off all timers with a more complex input so as to not accidentally do it.  Ideally, I would extend this system with a more modular input method.  I want to include keypad entry like on microwaves, an easier slider input for those who cannot twist knobs or input on small keys, and ideally even voice input.  The customizability of this is not shown in the proof of concept, but the code framework can certainly support it.

Files + Code + Video

Final Project Idea

I have an idea on a final project, please let me know if you have any advice or feedback- I would greatly appreciate it!

I want to create a smart watch interface for someone with anxiety and panic disorder. I plan on using real-time biometric data from sensors and using the data to trigger and display things using p5.js.

The watch has two modes: Normal Mode & Panic Mode.Normal Mode includes a watch interface that displays the time and date, in addition to the sensor data in an artistic, data-visualization way (I am thinking something similar to a mood visualizer type of thing). The panic mode can be triggered through two ways: a panic button the user presses or sensor data that indicates the user is having a panic attack. In Panic Mode, the canvas cycles through the following anxiety relieving techniques:

  1. Deep Breathing Exercise: using calming graphics to help guide the user through a deep breathing exercise. I will use online resources to figure out how the breathing exercise need to be in order to work, like WebMed’s Techiques for deep breathing.
  2. Body Scan: using the body scan technique found here.
  3. Distraction/Game Technique: using a jigsaw puzzle or some sort of mind occupying game that reduces stress but still allow you to channel your overactive brain somewhere.
  4. 5 Senses Technique: using the 5 senses to ground you, as shown below:
This is a type of grounding technique to help bring you back to reality during a panic attack.

If all of the following techniques do not work, then this triggers a “call emergency contact” state, which calls someone you designated as a person to reach out to. For example, “calling your mom…”

The biometric sensors I am thinking of using are: a heart rate (PPG) sensor, a GSR sensor, and a respiratory rate sensor. The last one, I might not need, I am waiting to confirm with a specialist…

The photoplethysmography (PPG) circuit ascertains the user’s heart rate.
The galvanic skin response (GSR) circuit ascertains the user’s skin conductance level – a measurement loosely coupled with perspiration indicative of stressful conditions (in other words, the more stressed you are, the more you sweat).

Assignment 8: Fan-Powered Door Chimes

State Machine: Distance Door Bells

Problem: Door bells are usually intrusive noises. They are sudden and loud and usually disturb the peace, or at least peace hoped for, in your home. In addition, most door bells only have two states: at the door or not. Because of this, guests usually must wait at the door for the host to arrive at the door and the host usually must drop everything they’re doing immediately to go and get the door. Is there a way to create a simple doorbell that does not alarm everyone in the house and also gives you a sense of where people are relative to the door?

General solution: A door chime system that created more calming/less disturbing sounds in order to alert the host. The system can also relay more information, namely, how far away someone is from the door to symbolize how much time the host has to finish whatever they are doing before making their guest wait at the door. As a guest gets closer and closer to the door, the chimes will grow louder and louder. Also, doorbells are traditionally known, at least at my house, for alarming dogs and setting them into a fit. To avoid this, these chimes can also be turned to a “do not disturb” mode for when people aren’t home or are asleep, so they chimes do not make noise to disturb the peace.

Proof of Concept: My system utilized the following components:

  • Ultrasonic Sensor: used to measure distance to simulate guest proximity to door
  • Fan: the driver of the chimes, causing them to knock together and create noise depending on the fan speed determined by the distance sensor
    • Transistor: acts as a voltage gate to allow the fan to be controlled at various speeds
    • Fan has 4 different states of sound production: just fan noise with no air movement, light air with light chimes, medium air with medium chimes, heavy air with heavy chimes
  • Push button: acts as an interrupt that changes the “mode” of the doorbell

Assignment8_FanDoorChimes

Files above, but after potentially shorting Chance’s computers and digging deeper into my own arduino issues… I am going to skip the video portion to get those things figured out for now.

Assignment 7: Kitchen Bells

State Machine: Ovens

Problem: While cooking, a chef must keep an eye on two different temperatures in order to make sure what put in an oven is cooked through. The oven temperature is easy enough, most ovens print their temperature on an LCD screen and some even “beep” when it reaches a target temperature. That’s great, but the more important number, one could argue, is the internal temperature of whatever is in the oven. How can someone keep track of that temperature without opening the oven to check/disturbing the oven temperature?

General solution: A (5 year from now quality) temperature sensor that transmits the food’s internal temperature and translates its “done-ness” not through numbers, but through sound frequency. Sound would potentially allow for a better gradient for understanding where your food is at in its cooking progression and it would also allow chefs to know the status at all times even if they leave the room or start looking at other things in the kitchen. 

Proof of Concept: My system utilized the following components:

  • A temperature sensor: used to read ambient temperature in the demo, but symbolizes the internal temperature of a food
  • Servo: acts as output mechanism to create sound patterns to relay information about food’s cooking state
    • Right now, I have the delays set up based on different ranges between the “final temperature” of your food and its “current temperature”. For the servo, it makes the most sense because it would be pretty difficult to distinguish between single digits on angles of rotation (frequency) as opposed to how often it oscillates between two points. If I would have used a speaker, I probably would have set it up so that the frequency/pitch of the sound generated by the temperature difference acts as a gradient because it is relatively easier to tell if a sound is getting higher or lower-pitched (even for the musically-challenged like myself). 
  • Push button: acts as an interrupt that signifies opening the oven, thus alerting the user that something is wrong (like they left the oven open or someone else checked)

Further Development: With the servo, if I would have had fewer things to work on this weekend, you could also turn this system into a set of chimes so that you aren’t working with mechanical sounds as an output. You could string pieces of metal or glass or other things around the servo and they could hit each other to produce more delicate/harsh/whatever-kind-of-sound-you-want-for-your-home sounds.

Assignment7

Crit 1: Cycling Tire Monitoring System

Problem

In the manufacturing of physical goods, it is often difficult to test for small defects. In the case of products such as rubber cycling tubing, small, hard to detect perforations can become much more troublesome for clients in the lifecycle of the product. Additionally, it can be difficult for active cyclists to focus on identifying non major leaks and gradual changes in tire pressure on long rides.

Solution

A mounted sensor array focused on detecting both leak frequencies and changes in tire pressure can be used to streamline the tire quality assurance process, and help signal the need for tire patching or tubing replacement on the fly for cyclists. Using microphones to pick up sounds within common frequencies for leaks, as well as using an air pressure sensor to track significant changes from an ideal benchmark can be used in concert with visual indicators to help identify tears and deformations.

crit1

Proof of Concept

The sensor array would have a visual indicator tied to each sensor to attempt to give users an idea of where a leak would be happening, or if tire pressure was being lost. After attempting to use an LCD to provide descriptive error messaging, I decided to use a series of LCDs in concert with microphones to simulate air pressure leaks, as well as a flex sensor to simulate an air pressure sensor.

Crit1

Concert Buddy

Problem

Does loud music damage hearing?

At concerts, the music playing is so loud that you can’t communicate with your friends. You’re feeling claustrophobic and you want to tell your friend that you want to leave, take a break and get some water. Unfortunately, you’ve virtually lost your voice from screaming every song and over the booming music its hard to speak to them. What do you do? It is also hard to find a balance in terms of getting a place near the front and also enjoying the concert at a volume that is safe for your ears.

Solution:

A wearable that can tell you options based on your current situation. One that can be worn to loud situations like parties/concert or even just construction area.

Proof Of Concept:

A device with preloaded responses and a LCD screen.  Each option is associated with a  color which indicates the  immediacy/importance of the message. The potentiometer can be used to select the message. Additionally, the device has a sound detector. The sound detector measures the volume level of the surroundings and illuminates a blue light, alerting the user that the sound level is high enough to be damaging to the ears.

Fritzing and Code

Link to Video