Problem
It can be very difficult for people with seeing difficulty to orient themselves. In fact, it is already difficult enough to locate the braille signs, which is supposed to help them. In addition, there are situations where visual clues are necessary, such as deciding which elevator goes up when two arrive at the same time. Here is a video that mentions such difficulty.
As a result, people with visual difficulty have to memorize routes. It is also very challenging for them to travel through a new district or locate a room in an unfamiliar building.
Solution
The key that causes this problem is the lost of visual clues. When we walk into a new building, we can look for signs for directions. When we travel through a new place, we can use google map as a guide. However, for people with seeing difficulty, the same clues are either impossible to get, like the road sings for example, or very inaccessible like the braille sign that hides behind a door.
To solve this problem, I would like to translate the visual clues into braille on a device that is accessible. In other words, instead of looking for physical sign of braille, the user can read the signs in braille on a device in his or her hand. This device, however, can not only be used as a navigator but also a “seeing” tool to read the visuals in a space or environment.
As shown in the figure above, the device have 12 digits for displaying letters and punctuations. Each digit is made of 6 solenoids and will display alphabet in braille. The rotary encode on the right is for flipping to the next 12 letters in the sentences. There are also a confirm button and a back button. This device can be connected to smart phone and indoor devices through Bluetooth or wi-fi.
Suppose a person with difficulty wants to go to new place, he or she can use app on his or her smartphone to determine the route. The person can then use the device to navigate while walking. When there are road signs or alerts that need attention, the device will vibrate and display the message. The device will go back to the route when the alert was read and confirm button pressed. The person can also use the back button to check the last direction or instruction.
Of course, the effectiveness of this device depends heavily on how small we can make it to be. With the current material of mini-solenoid, I can make it to have a length of 18 cm and a width of 7.2 cm. This cannot work effective as a navigation tool because it is too big and cumbersome. However, it can still be used while stationary such as in a restaurant, museum, and a room.
If we have the interactive surface technology in the future, this is the design I will give to the device. It wrap around the arm of the user so that the user can hold a white cane in one hand, and read the information using another hand. No more buttons and rotary encoder are needed because everything can be displayed and interacted with through the surface.
Proof of Logic
The demo is made up of a solenoid, a Bluetooth module, 6 LED, 4 buttons, and a vibration motor. The LEDs represent the 6-dot grid of the braille. The corresponding grid is shown below.
Because the demo-device is relatively complex, it will be explained with video of demonstration.
This shows the normal function of the solenoid. In the later demonstration it does not function correctly due to the insufficient current arduino UNO supplies.
The two button pressed in the video represents the rotary encode that allows the user to flip to the next or previous 12 letters.
The buttons pressed in the video are confirm button and back button, which allows the user to display the next or previous message instruction.
Finally, in the video alerts are sent from the PC to the device through Bluetooth. The device then vibrate and display the alert with the corresponding braille letter.
Schematic:
Code:
//Defining the pins #define solenoide 12 #define motor 13 #define rotary_up 5 #define rotary_down 4 #define confirm 3 #define back 2 #define LED1 11 #define LED2 10 #define LED3 9 #define LED4 8 #define LED5 7 #define LED6 6 //Variables //constant variables char *INPUT_STRINGS[] = {"information destk is at first floor","elevator is to the right by ten meters" ,"this is room b ten"}; //Message is like the route and direction int message; int message_length; String string; int start_letter; //Alert is like the signs or alerts on the street or in buildings String incoming_alert; //string that stores the incoming message String alert[5]; int alert_index; bool has_alert; bool vibration_on; //variable for interrupt, note that message is also a ISR variable const bool isInterrupt = true; int message_number = 3; //timer //Clock 1 is the timer for checking the rotary encoder unsigned long clock1 = 0; // variable for timing const int INTERVAL1 = 300; // milliseconds between updates //Clock 2 is for serial printing for illustration unsigned long clock2 = 0; // variable for timing const int INTERVAL2 = 2000; // milliseconds between updates //Clock 3 is for serial printing for illustration unsigned long clock3 = 0; // variable for timing const int INTERVAL3 = 1500; // milliseconds between updates // the interrupt method // NOTE: we shouldn't use Serial.prinln(), delay(), and many // other functions in an interrupt method void ConfirmSwitchPressed() { if (digitalRead(confirm) == LOW) { if (has_alert == true) { //alert index also show how many alerts are in the alert array if (alert_index == 1) { has_alert = false; } for (int i; i < alert_index; i++) { alert[i] = alert[i + 1]; } if (alert_index != 0) { alert_index -= 1; } Serial.println("Alert read"); return; } if (has_alert == false) { message == message_number ? message = 0 : message++; start_letter = 0; Serial.println("Next Message"); } } } void BackSwitchPressed() { if (digitalRead(back) == LOW) { if (has_alert == false) { message == 0 ? message = message_number : message--; start_letter = 0; Serial.println("Previous Message"); } } } //Functions //demonstration function that shows how to move solenoid void move_solenoid() { digitalWrite(solenoide, HIGH); delay(1000); digitalWrite(solenoide, LOW); delay(1000); } //Demonstration function void vibrate_motor() { digitalWrite(motor, HIGH); delay(2000); digitalWrite(motor, LOW); delay(2000); } void show_letters() { if (has_alert == true) { braille(alert[0][start_letter]); } if (has_alert == false) { braille(INPUT_STRINGS[message][start_letter]); } //if there are actually 12 digits, each made of 6 solenoid //int digit; //for (i=start_letter, i< start_letter +13,i++) { // braille(digit, INPUT_STRINGS[message][start_letter + i]) // digit += 1; // } } //This function put the incoming alert into the alert array void insert_alert() { //-1 indicates that there is no \ in the alert message int null_index = incoming_alert.indexOf('.'); if (null_index != -1 and alert_index < 4) { alert[alert_index] = incoming_alert.substring(0, null_index); alert_index += 1; //Tell the system there are alerts has_alert = true; //Turn the vibration and its corresponding timer on digitalWrite(motor, HIGH); vibration_on = true; clock3 = millis() + INTERVAL3; //Clear the string for the next incoming alert incoming_alert = ""; Serial.print("Alert received, "); Serial.println("vibration on."); } } //In the actual desgin instead of this demo, the 6 LEDs are 6 solenoides void braille(char input) { digitalWrite(LED1, LOW); digitalWrite(LED2, LOW); digitalWrite(LED3, LOW); digitalWrite(LED4, LOW); digitalWrite(LED5, LOW); digitalWrite(LED6, LOW); digitalWrite(solenoide, LOW); if (input == 'a') { digitalWrite(LED1, HIGH); digitalWrite(solenoide, HIGH); } if (input == 'b') { digitalWrite(LED1, HIGH); digitalWrite(LED2, HIGH); digitalWrite(solenoide, HIGH); } if (input == 'c') { digitalWrite(LED1, HIGH); digitalWrite(LED4, HIGH); digitalWrite(solenoide, HIGH); } if (input == 'd') { digitalWrite(LED1, HIGH); digitalWrite(LED4, HIGH); digitalWrite(LED5, HIGH); digitalWrite(solenoide, HIGH); } if (input == 'e') { digitalWrite(LED1, HIGH); digitalWrite(LED5, HIGH); digitalWrite(solenoide, HIGH); } if (input == 'f') { digitalWrite(LED1, HIGH); digitalWrite(LED2, HIGH); digitalWrite(LED5, HIGH); digitalWrite(solenoide, HIGH); } if (input == 'g') { digitalWrite(LED1, HIGH); digitalWrite(LED2, HIGH); digitalWrite(LED4, HIGH); digitalWrite(LED5, HIGH); digitalWrite(solenoide, HIGH); } if (input == 'h') { digitalWrite(LED1, HIGH); digitalWrite(LED2, HIGH); digitalWrite(LED5, HIGH); digitalWrite(solenoide, HIGH); } if (input == 'i') { digitalWrite(LED2, HIGH); digitalWrite(LED4, HIGH); } if (input == 'j') { digitalWrite(LED2, HIGH); digitalWrite(LED4, HIGH); digitalWrite(LED5, HIGH); } if (input == 'k') { digitalWrite(LED1, HIGH); digitalWrite(LED3, HIGH); digitalWrite(solenoide, HIGH); } if (input == 'l') { digitalWrite(LED1, HIGH); digitalWrite(LED2, HIGH); digitalWrite(LED3, HIGH); digitalWrite(solenoide, HIGH); } if (input == 'm') { digitalWrite(LED1, HIGH); digitalWrite(LED3, HIGH); digitalWrite(LED4, HIGH); digitalWrite(solenoide, HIGH); } if (input == 'n') { digitalWrite(LED1, HIGH); digitalWrite(LED5, HIGH); digitalWrite(LED3, HIGH); digitalWrite(LED4, HIGH); digitalWrite(solenoide, HIGH); } if (input == 'o') { digitalWrite(LED1, HIGH); digitalWrite(LED5, HIGH); digitalWrite(LED3, HIGH); digitalWrite(solenoide, HIGH); } if (input == 'p') { digitalWrite(LED1, HIGH); digitalWrite(LED2, HIGH); digitalWrite(LED3, HIGH); digitalWrite(LED4, HIGH); digitalWrite(solenoide, HIGH); } if (input == 'q') { digitalWrite(LED1, HIGH); digitalWrite(LED2, HIGH); digitalWrite(LED3, HIGH); digitalWrite(LED4, HIGH); digitalWrite(LED5, HIGH); digitalWrite(solenoide, HIGH); } if (input == 'r') { digitalWrite(LED1, HIGH); digitalWrite(LED2, HIGH); digitalWrite(LED3, HIGH); digitalWrite(LED5, HIGH); digitalWrite(solenoide, HIGH); } if (input == 's') { digitalWrite(LED2, HIGH); digitalWrite(LED3, HIGH); digitalWrite(LED4, HIGH); } if (input == 't') { digitalWrite(LED2, HIGH); digitalWrite(LED3, HIGH); digitalWrite(LED4, HIGH); digitalWrite(LED5, HIGH); } if (input == 'u') { digitalWrite(LED1, HIGH); digitalWrite(LED3, HIGH); digitalWrite(LED6, HIGH); digitalWrite(solenoide, HIGH); } if (input == 'v') { digitalWrite(LED1, HIGH); digitalWrite(LED2, HIGH); digitalWrite(LED3, HIGH); digitalWrite(LED6, HIGH); digitalWrite(solenoide, HIGH); } if (input == 'w') { digitalWrite(LED2, HIGH); digitalWrite(LED6, HIGH); digitalWrite(LED4, HIGH); digitalWrite(LED5, HIGH); } if (input == 'x') { digitalWrite(LED1, HIGH); digitalWrite(LED3, HIGH); digitalWrite(LED4, HIGH); digitalWrite(LED6, HIGH); digitalWrite(solenoide, HIGH); } if (input == 'y') { digitalWrite(LED1, HIGH); digitalWrite(LED6, HIGH); digitalWrite(LED3, HIGH); digitalWrite(LED4, HIGH); digitalWrite(LED5, HIGH); digitalWrite(solenoide, HIGH); } if (input == 'z') { digitalWrite(LED1, HIGH); digitalWrite(LED6, HIGH); digitalWrite(LED3, HIGH); digitalWrite(LED5, HIGH); digitalWrite(solenoide, HIGH); } if (input == ',') { digitalWrite(LED2, HIGH); } if (input == '.') { digitalWrite(LED2, HIGH); digitalWrite(LED6, HIGH); digitalWrite(LED5, HIGH); } if (input == '!') { digitalWrite(LED2, HIGH); digitalWrite(LED3, HIGH); digitalWrite(LED5, HIGH); } } void setup() { // Setup the pin modes pinMode(solenoide, OUTPUT); pinMode(motor, OUTPUT); pinMode(LED1, OUTPUT); pinMode(LED2, OUTPUT); pinMode(LED3, OUTPUT); pinMode(LED4, OUTPUT); pinMode(LED5, OUTPUT); pinMode(LED6, OUTPUT); //When the buttons are pushed, digitalRead shows 0 pinMode(rotary_up, INPUT_PULLUP); pinMode(rotary_down, INPUT_PULLUP); pinMode(confirm, INPUT_PULLUP); pinMode(back, INPUT_PULLUP); //Initiate the serial monitor Serial.begin(9600); // attach the interrupt pin to a method if (isInterrupt) { attachInterrupt (digitalPinToInterrupt(confirm), ConfirmSwitchPressed, FALLING); attachInterrupt (digitalPinToInterrupt(back), BackSwitchPressed, FALLING); } } void loop() { //Determining the number of messages/alerts and the length of each message if (has_alert == true) { string = alert[0]; message_length = string.length(); } if (has_alert == false) { string = INPUT_STRINGS[message]; message_length = string.length(); } //Receiving the characters of incoming alerts while (Serial.available()) { //while there is data available on the serial monitor //Serial.println(Serial.read()); incoming_alert += char(Serial.read()); //store string from serial command } insert_alert(); //Check if any movement has been made on the rotary encoder if (millis() >= clock1) { if (digitalRead(rotary_up) == LOW) { if (start_letter + 12 <= message_length) { start_letter += 12; Serial.println("Next 12 letters"); } } if (digitalRead(rotary_down) == LOW) { if (start_letter - 12 >= 0) { start_letter -= 12; Serial.println("Last 12 letters"); } } clock1 = millis() + INTERVAL1 ; } show_letters(); //For serial printing of demonstration if (millis() >= clock2) { if (incoming_alert != "") { Serial.print("Incoming Alert: "); Serial.println(incoming_alert); } //Serial.print("The current message or alert: "); Serial.println(string); clock2 = millis() + INTERVAL2 ; } //For 2-second duration of the vibration motor if (millis() >= clock3 and vibration_on == true) { digitalWrite(motor, LOW); vibration_on = false; Serial.println("vibration off."); } }