It comes with two parts: a Joystick and a button. The joystick is for a simple answer, and the button is for inquiries.
The joystick corresponds with one’s positivity in the answer in counterclockwise order, following down (no), right (maybe), up (yes), left (sounds great). The pitch of the sound corresponds as well, from a lower to higher pitch depending on the positivity in the answer.
Down; C
Right; DUp; ELeft; F
The button shows one’s inquiry in order. The user can change the message as well, but you can assign each note that corresponds with your inquiries.
DefaultPressed once; GPressed twice; APressed three times; BPressed four times; next octave C
Q&A
What if I am not a perfect pitch?
Does the pitch of notes corresponding with your answer help? Does LCD? Please let me know!
Schematics:
//Codes
#include <Wire.h>
#include <LiquidCrystal.h>
LiquidCrystal lcd(13, 12, 5, 4, 3, 2);
int speaker = 7;
const int button = 6;
int val;
int counter;
int pressed;
void setup() {
lcd.begin(16,2);
pinMode(speaker, OUTPUT);
pinMode(button, INPUT_PULLUP);
Serial.begin(9600);
lcd.print("Default");
}
void loop() {
val = digitalRead(button);
int sensorReading = analogRead(A2);
int sensorReading2 = analogRead(A3);
if (sensorReading < 300) {
lcd.setCursor(0,0);
lcd.print("Yes. ");
tone(speaker,659);
delay(500);
noTone(speaker);
}
if (sensorReading > 800) {
lcd.setCursor(0,0);
lcd.print("No. ");
tone(speaker,(523));
delay(500);
noTone(speaker);
}
if (sensorReading2 <300) {
lcd.setCursor(0,0);
lcd.print("Maybe. ");
tone(speaker, 587);
delay(500);
noTone(speaker);
}
if (sensorReading2 > 800) {
lcd.setCursor(0,0);
lcd.print("Sounds great. ");
tone(speaker, 698);
delay(500);
noTone(speaker);
}
if (val == LOW){
pressed=counter ++;
delay(250);
Serial.println(pressed);
if (pressed == 0){
lcd.setCursor(0,0);
lcd.print("I need water.");
tone(speaker, 784);
delay(500);
noTone(speaker);
}
if (pressed == 1){
lcd.setCursor(0,0);
lcd.print("I want food. ");
tone(speaker, 880);
delay(500);
noTone(speaker);
}
if (pressed == 2){
lcd.setCursor(0,0);
lcd.print("Can you bring");
lcd.setCursor(0,2);
lcd.print("my phone?");
tone(speaker, 988);
delay(500);
noTone(speaker);
}
if (pressed == 3){
lcd.setCursor(0,0);
lcd.print("I need to be ");
lcd.setCursor(0,2);
lcd.print("alone. ");
tone(speaker, 1046);
delay(500);
noTone(speaker);
}
}
}
Context: One of the main contributions of technology is mediation: technology mediates the agency of both disabled and fully enabled humans in the world, rebuilding their relation with the environment.
Problem: For visually impaired people, many applications have been developed that scan and recognise objects in space for visually impaired people. This way, the user can perceive spatial information without employing touch. The visual information is translated into auditory words, which describe an object or a spatial condition.
This method facilitates the daily life for people with visual impairment. However, it is inadequate when it comes to the perception of art. A colourful painting cannot be described through a sequence of colour matches that may bear no meaning to people with congenital vision disability.
Solution: ‘Hearing the colours is a wearable interface that turns the experience of colours and visual art into an audio performance. The wearable prompts the user to move their hand and ‘grasp’ the sounds of the colours in a similar way with that they execute to ‘grasp’ the Braille dots for reading.
The interface plays the role of a sensory substitution device (SSD). After practice, the user learns to stimulate spontaneously the experiential quality of “seeing a color” through a new set of sensoricognitive skills. . This changes the classical definition of sensory modalities and contributes to the emergence of a form of “artificial synaesthesia”.
The interface is designed to transform the static observation of art into an interplay of visuals, sound and body movement. However, hearing the colors through a wearable device should not only be constrained to the spectrum of artwork. It contains the potential to turn the body into a scanning device, that moves and interacts with the space itself, even with other human beings.
IdeaPerceiving the colours in artPerceiving people through their skin colour
Color as a R:G:B and H:S:B code
Computational Logic:
Demonstration Demo:
Circuit design:
Code:
#include <Wire.h>
#include "Adafruit_TCS34725.h"
/* Initialise with specific int time (700ms) and gain(1x) values
Integration time = how often it reads the colors
Longer integration times can be used for increased sensitivity at low light levels.
Valid integration times are: 2.4MS, 24MS, 50MS, 101MS, 154MS, 700MS
Sets the gain of the ADC to control the sensitivity of the sensor.
Valid gain settings are: 1x(no gain), 4x, 16x, 60x
*/
Adafruit_TCS34725 myRGB = Adafruit_TCS34725(TCS34725_INTEGRATIONTIME_50MS, TCS34725_GAIN_4X);
int R = 0;
int G = 0;
int B = 0;
int hsv [3];
const int speaker = 11;
int pitch; int duration; int delayT;
/*********************************************************************************************/
void setup() {
Serial.begin(9600);
if (myRGB.begin()) {
Serial.println("The colorSensor is found");
} else {
Serial.println("No colorSensori is found");
while (1);
}
Serial.println("*****************************************************");
Serial.println();
}
/*********************************************************************************************/
void loop() {
getRGBValues();
rgb2Hsv(R, G, B, hsv);
colors2Sounds();
tone(speaker, pitch, duration);
delay(delayT);
} // loop
/*********************************************************************************************/
void colors2Sounds() {
if (hsv[0] <= 1) {
pitch = 100;
duration = 400;
delayT = 0;
Serial.println("red");
} else if (hsv[0] <= 5) {
pitch = 300;
duration = 400;
delayT = 50;
Serial.println("purple");
} else if (hsv[0] <= 13) {
pitch = 500;
duration = 400;
delayT = 100;
Serial.println("orange");
} else if (hsv[0] <= 17) {
pitch = 1000;
duration = 400;
delayT = 180;
Serial.println("yellow");
} else if (hsv[0] <= 25) {
pitch = 1800;
duration = 400;
delayT = 230;
Serial.println("green");
} else if (hsv[0] <= 33) {
pitch = 2400;
duration = 400;
delayT = 300;
Serial.println("greenish blue");
} else if (hsv[0] <= 55) {
pitch = 3000;
duration = 400;
delayT = 400;
Serial.println("blue");
}
}
void getRGBValues() {
float _red, _green, _blue;
myRGB.setInterrupt(false);
delay(60);
myRGB.getRGB(&_red, &_green, &_blue);
myRGB.setInterrupt(true);
//Print RGB values
/* Serial.print("Red: "); Serial.print(_red); Serial.print(" *** ");
Serial.print("Green: "); Serial.print(_green); Serial.print(" *** ");
Serial.print("Blue: "); Serial.print(_blue); Serial.print(" *** ");
Serial.println(); */
R = _red;
G = _green;
B = _blue;
} // getRGBValues
void rgb2Hsv(byte r, byte g, byte b, int hsv[]) {
float rd = (float) r / 255;
float gd = (float) g / 255;
float bd = (float) b / 255;
float max = threeway_max(rd, gd, bd), min = threeway_min(rd, gd, bd);
float h, s, v = max;
float d = max - min;
s = max == 0 ? 0 : d / max;
if (max == min) {
h = 0; // achromatic
} else {
if (max == rd) {
h = (gd - bd) / d + (gd < bd ? 6 : 0);
} else if (max == gd) {
h = (bd - rd) / d + 2;
} else if (max == bd) {
h = (rd - gd) / d + 4;
}
h /= 6;
}
hsv[0] = int(h * 100);
hsv[1] = int(s * 100);
hsv[2] = int(v * 100);
Serial.println((String)"Hue: " + hsv[0] + " Saturation: " + hsv[1] + " Value: " + hsv[2]);
}
float threeway_max(float r, float g, float b) {
float res;
res = r;
if (g > res) {
res = g;
}
if (b > res) {
res = b;
}
return res;
}
float threeway_min(float r, float g, float b) {
float res;
res = r;
if (g < res) {
res = g;
}
if (b < res) {
res = b;
}
return res;
}
Generate sounds related to an environment that make the environment easier to use for people who might have issue with collecting data via sight or physical capabilities. This includes data they might not be able to perceive, say “how many people are in the elevator coming to my floor?” or “when will my bus arrive at my bus stop?”
Universal design and accessibility are good starting places. I work with a lot of heat when I’m in the studio, welding metal or working with glass. As we usually say (rather loudly) while going to the first-aid box, “hot metal looks just like cold metal” or “hot glass looks just like cold glass”. I have temperature probes but they require me using both hands while looking at a screen. How can I get an audible warning that I’m about to touch something dangerously hot?
Kraftwerk:
Tour de France, use of bicycle/human sounds as part of a song: https://www.youtube.com/watch?v=rTe7U92ecX8
Autobahn, which I talked about but we didn’t listen: https://www.youtube.com/watch?v=1DO-Ddqbqgs
Art of Noise is another band that used sampling to bring non-instrument noises to music:
Another song using the orchestral hit, Close (To the Edit): https://www.youtube.com/watch?v=-sFK0-lcjGU
German industrial band Einstürzende Neubauten uses found objects for percussion and background tambre: https://www.youtube.com/watch?v=KrfOahWNT_A
Cultural conflation
M.I.A. has western/asian drifter culture as part of a video shot in Morrocco in support of a movement to let women legally drive cars. https://www.youtube.com/watch?v=3Yuqxl284cg
Environmental sound
why is this machine so quiet?
what is all the computation happening in this machine?
how is this different from a Trader Joe’s microwave-meal-box?
Colorblind simulator
Maria asked a question about how to express color as sound. One place to start this sort of research is a colorblind simulator: https://www.color-blindness.com/coblis-color-blindness-simulator
nut allergy: Co-living with people who just met you, makes it difficult to always remember about your food allergies. If not properly cleaned, some of the ingredients you are allergic to, may have come into contact with surfaces. Sound production when nuts have touched specific areas.
When you are stressed, house should automatically produce calm sounds or light condition, that could help you calm down and force you into short breaks from studying.
Never been locked out by your flatmates. Notify if everybody is home/returned, so that people know when to lock the door.
Elevators in my tall apartment building (22 floors) can take a long time to arrive because we’re limited to 1 household per elevator trip. If there are multiple people waiting for elevators during ‘rush hour’ (and yes there is still rush hour during covid times, many people walk their dogs at the same time) it’d be good if a chirping sound was made before I’m about to leave the house.
The recycling area in my apartment building is usually overflowing; I wish that a sound would be made if it’s empty so I can immediately bring my recyclables over.
It can be hard to stay on top of perishables in the fridge, both stocking what I need (e.g. eggs, bananas, milk etc.) and making sure that I consume food in an orderly manner (First in, First out) so if the fridge could make a sound to remind me if I need to replenish something or if I need to consume something soon.
My white whale in cooking is making toast in my oven. Unlike boiling / frying, there is little sensory feedback (smell, sound, sight, etc.) on how hot the burner is and I often end up burning my bread. I wish that the oven would tell me once it was done.
I have a humidifier and air purifier – I wish they would let me know when their filters needed changing.
When my keys are not in their appropriate storage place, I wish that they would make a sound to remind me to place them back.
If I wake up and we might have a few hours of sunshine that day, I wish my home would encourage me to go outside by playing a spritely tune when I get out of bed in the morning.
If I haven’t meditated that day, I wish my meditation chair could make a sound to call this to my attention when my roommate isn’t on work calls.
a song clothes washer/dryer plays when the cycles are finished. They are in a closet so sometimes they go unnoticed even though the cycles are finished.
Some sound effect that plays before I head out to let me know if it is raining or snowing outside. It is very frustrating to realize I need to go back up to grab an umbrella when I already went all the way down.
Sound effect that reminds me that the defrost is finished in the sink. It usually takes about 90 minutes and I can forget about it from time to time.
Plays a very happy song when package arrives. Sometimes I don’t know they are already here because they are in the leasing office.
A voice line to remind me to head out 5 minutes before the bus arrives. There is a bus stop near my apartment and it would be nice if I can get to the stop right before the bus comes.
What sounds would you add to your house to make it more accessible?
I sometimes plug in the phone/tablet/laptop charger, but not to the devices, so when I wake up in the morning, things are out of battery. A sound that reminds me to plug in the devices would be helpful.
When filling water in a rush, I leave the kettle empty, and when I need water again later, I have to wait for the water to boil. A sound that reminds me to fill up the kettle befoe I need water should help.
Lights that are accidentally left on over night!
This one is far from realistic, but finding things within the house is not easy! Especially when it involves climbing up the stairs… so if somehow the house can know where things are and make sounds…
What sounds would you add to your house to make it more accessible?
Pretty much every gadgets in my house talk/play sound when something happens (even the cat feeder..or the toilet..) but one thing I wish is my studio room!! As you can tell from my past projects I have many issues with my studio regarding my health! Wouldn’t it be nice for my studio room to have a detector that tells me how toxic the air level is right now..
Or I want some sound that tells me how the water temperature is just right when I shower. Like when the water’s temperature reaches the user’s set temperature, it makes a sound.
My dad tends to fall asleep while he is boiling something on the stove, and it sometimes overflows. He says that he wishes that the sound could tell him that the stove is overheating.
