Problem
It can be very difficult for people with seeing difficulty to orient themselves. In fact, it is already difficult enough to locate the braille signs, which is supposed to help them. In addition, there are situations where visual clues are necessary, such as deciding which elevator goes up when two arrive at the same time. Here is a video that mentions such difficulty.
As a result, people with visual difficulty have to memorize routes. It is also very challenging for them to travel through a new district or locate a room in an unfamiliar building.
Solution
The key that causes this problem is the lost of visual clues. When we walk into a new building, we can look for signs for directions. When we travel through a new place, we can use google map as a guide. However, for people with seeing difficulty, the same clues are either impossible to get, like the road sings for example, or very inaccessible like the braille sign that hides behind a door.
To solve this problem, I would like to translate the visual clues into braille on a device that is accessible. In other words, instead of looking for physical sign of braille, the user can read the signs in braille on a device in his or her hand. This device, however, can not only be used as a navigator but also a “seeing” tool to read the visuals in a space or environment.
As shown in the figure above, the device have 12 digits for displaying letters and punctuations. Each digit is made of 6 solenoids and will display alphabet in braille. The rotary encode on the right is for flipping to the next 12 letters in the sentences. There are also a confirm button and a back button. This device can be connected to smart phone and indoor devices through Bluetooth or wi-fi.
Suppose a person with difficulty wants to go to new place, he or she can use app on his or her smartphone to determine the route. The person can then use the device to navigate while walking. When there are road signs or alerts that need attention, the device will vibrate and display the message. The device will go back to the route when the alert was read and confirm button pressed. The person can also use the back button to check the last direction or instruction.
Of course, the effectiveness of this device depends heavily on how small we can make it to be. With the current material of mini-solenoid, I can make it to have a length of 18 cm and a width of 7.2 cm. This cannot work effective as a navigation tool because it is too big and cumbersome. However, it can still be used while stationary such as in a restaurant, museum, and a room.
If we have the interactive surface technology in the future, this is the design I will give to the device. It wrap around the arm of the user so that the user can hold a white cane in one hand, and read the information using another hand. No more buttons and rotary encoder are needed because everything can be displayed and interacted with through the surface.
Proof of Logic
The demo is made up of a solenoid, a Bluetooth module, 6 LED, 4 buttons, and a vibration motor. The LEDs represent the 6-dot grid of the braille. The corresponding grid is shown below.
Because the demo-device is relatively complex, it will be explained with video of demonstration.
This shows the normal function of the solenoid. In the later demonstration it does not function correctly due to the insufficient current arduino UNO supplies.
The two button pressed in the video represents the rotary encode that allows the user to flip to the next or previous 12 letters.
The buttons pressed in the video are confirm button and back button, which allows the user to display the next or previous message instruction.
Finally, in the video alerts are sent from the PC to the device through Bluetooth. The device then vibrate and display the alert with the corresponding braille letter.
Schematic:
Code:
//Defining the pins
#define solenoide 12
#define motor 13
#define rotary_up 5
#define rotary_down 4
#define confirm 3
#define back 2
#define LED1 11
#define LED2 10
#define LED3 9
#define LED4 8
#define LED5 7
#define LED6 6
//Variables
//constant variables
char *INPUT_STRINGS[] = {"information destk is at first floor","elevator is to the right by ten meters" ,"this is room b ten"};
//Message is like the route and direction
int message;
int message_length;
String string;
int start_letter;
//Alert is like the signs or alerts on the street or in buildings
String incoming_alert; //string that stores the incoming message
String alert[5];
int alert_index;
bool has_alert;
bool vibration_on;
//variable for interrupt, note that message is also a ISR variable
const bool isInterrupt = true;
int message_number = 3;
//timer
//Clock 1 is the timer for checking the rotary encoder
unsigned long clock1 = 0; // variable for timing
const int INTERVAL1 = 300; // milliseconds between updates
//Clock 2 is for serial printing for illustration
unsigned long clock2 = 0; // variable for timing
const int INTERVAL2 = 2000; // milliseconds between updates
//Clock 3 is for serial printing for illustration
unsigned long clock3 = 0; // variable for timing
const int INTERVAL3 = 1500; // milliseconds between updates
// the interrupt method
// NOTE: we shouldn't use Serial.prinln(), delay(), and many
// other functions in an interrupt method
void ConfirmSwitchPressed()
{
if (digitalRead(confirm) == LOW) {
if (has_alert == true) {
//alert index also show how many alerts are in the alert array
if (alert_index == 1) {
has_alert = false;
}
for (int i; i < alert_index; i++) {
alert[i] = alert[i + 1];
}
if (alert_index != 0) {
alert_index -= 1;
}
Serial.println("Alert read");
return;
}
if (has_alert == false) {
message == message_number ? message = 0 : message++;
start_letter = 0;
Serial.println("Next Message");
}
}
}
void BackSwitchPressed()
{
if (digitalRead(back) == LOW) {
if (has_alert == false) {
message == 0 ? message = message_number : message--;
start_letter = 0;
Serial.println("Previous Message");
}
}
}
//Functions
//demonstration function that shows how to move solenoid
void move_solenoid() {
digitalWrite(solenoide, HIGH);
delay(1000);
digitalWrite(solenoide, LOW);
delay(1000);
}
//Demonstration function
void vibrate_motor() {
digitalWrite(motor, HIGH);
delay(2000);
digitalWrite(motor, LOW);
delay(2000);
}
void show_letters() {
if (has_alert == true) {
braille(alert[0][start_letter]);
}
if (has_alert == false) {
braille(INPUT_STRINGS[message][start_letter]);
}
//if there are actually 12 digits, each made of 6 solenoid
//int digit;
//for (i=start_letter, i< start_letter +13,i++) {
// braille(digit, INPUT_STRINGS[message][start_letter + i])
// digit += 1;
// }
}
//This function put the incoming alert into the alert array
void insert_alert() {
//-1 indicates that there is no \ in the alert message
int null_index = incoming_alert.indexOf('.');
if (null_index != -1 and alert_index < 4) {
alert[alert_index] = incoming_alert.substring(0, null_index);
alert_index += 1;
//Tell the system there are alerts
has_alert = true;
//Turn the vibration and its corresponding timer on
digitalWrite(motor, HIGH);
vibration_on = true;
clock3 = millis() + INTERVAL3;
//Clear the string for the next incoming alert
incoming_alert = "";
Serial.print("Alert received, ");
Serial.println("vibration on.");
}
}
//In the actual desgin instead of this demo, the 6 LEDs are 6 solenoides
void braille(char input) {
digitalWrite(LED1, LOW);
digitalWrite(LED2, LOW);
digitalWrite(LED3, LOW);
digitalWrite(LED4, LOW);
digitalWrite(LED5, LOW);
digitalWrite(LED6, LOW);
digitalWrite(solenoide, LOW);
if (input == 'a') {
digitalWrite(LED1, HIGH);
digitalWrite(solenoide, HIGH);
}
if (input == 'b') {
digitalWrite(LED1, HIGH);
digitalWrite(LED2, HIGH);
digitalWrite(solenoide, HIGH);
}
if (input == 'c') {
digitalWrite(LED1, HIGH);
digitalWrite(LED4, HIGH);
digitalWrite(solenoide, HIGH);
}
if (input == 'd') {
digitalWrite(LED1, HIGH);
digitalWrite(LED4, HIGH);
digitalWrite(LED5, HIGH);
digitalWrite(solenoide, HIGH);
}
if (input == 'e') {
digitalWrite(LED1, HIGH);
digitalWrite(LED5, HIGH);
digitalWrite(solenoide, HIGH);
}
if (input == 'f') {
digitalWrite(LED1, HIGH);
digitalWrite(LED2, HIGH);
digitalWrite(LED5, HIGH);
digitalWrite(solenoide, HIGH);
}
if (input == 'g') {
digitalWrite(LED1, HIGH);
digitalWrite(LED2, HIGH);
digitalWrite(LED4, HIGH);
digitalWrite(LED5, HIGH);
digitalWrite(solenoide, HIGH);
}
if (input == 'h') {
digitalWrite(LED1, HIGH);
digitalWrite(LED2, HIGH);
digitalWrite(LED5, HIGH);
digitalWrite(solenoide, HIGH);
}
if (input == 'i') {
digitalWrite(LED2, HIGH);
digitalWrite(LED4, HIGH);
}
if (input == 'j') {
digitalWrite(LED2, HIGH);
digitalWrite(LED4, HIGH);
digitalWrite(LED5, HIGH);
}
if (input == 'k') {
digitalWrite(LED1, HIGH);
digitalWrite(LED3, HIGH);
digitalWrite(solenoide, HIGH);
}
if (input == 'l') {
digitalWrite(LED1, HIGH);
digitalWrite(LED2, HIGH);
digitalWrite(LED3, HIGH);
digitalWrite(solenoide, HIGH);
}
if (input == 'm') {
digitalWrite(LED1, HIGH);
digitalWrite(LED3, HIGH);
digitalWrite(LED4, HIGH);
digitalWrite(solenoide, HIGH);
}
if (input == 'n') {
digitalWrite(LED1, HIGH);
digitalWrite(LED5, HIGH);
digitalWrite(LED3, HIGH);
digitalWrite(LED4, HIGH);
digitalWrite(solenoide, HIGH);
}
if (input == 'o') {
digitalWrite(LED1, HIGH);
digitalWrite(LED5, HIGH);
digitalWrite(LED3, HIGH);
digitalWrite(solenoide, HIGH);
}
if (input == 'p') {
digitalWrite(LED1, HIGH);
digitalWrite(LED2, HIGH);
digitalWrite(LED3, HIGH);
digitalWrite(LED4, HIGH);
digitalWrite(solenoide, HIGH);
}
if (input == 'q') {
digitalWrite(LED1, HIGH);
digitalWrite(LED2, HIGH);
digitalWrite(LED3, HIGH);
digitalWrite(LED4, HIGH);
digitalWrite(LED5, HIGH);
digitalWrite(solenoide, HIGH);
}
if (input == 'r') {
digitalWrite(LED1, HIGH);
digitalWrite(LED2, HIGH);
digitalWrite(LED3, HIGH);
digitalWrite(LED5, HIGH);
digitalWrite(solenoide, HIGH);
}
if (input == 's') {
digitalWrite(LED2, HIGH);
digitalWrite(LED3, HIGH);
digitalWrite(LED4, HIGH);
}
if (input == 't') {
digitalWrite(LED2, HIGH);
digitalWrite(LED3, HIGH);
digitalWrite(LED4, HIGH);
digitalWrite(LED5, HIGH);
}
if (input == 'u') {
digitalWrite(LED1, HIGH);
digitalWrite(LED3, HIGH);
digitalWrite(LED6, HIGH);
digitalWrite(solenoide, HIGH);
}
if (input == 'v') {
digitalWrite(LED1, HIGH);
digitalWrite(LED2, HIGH);
digitalWrite(LED3, HIGH);
digitalWrite(LED6, HIGH);
digitalWrite(solenoide, HIGH);
}
if (input == 'w') {
digitalWrite(LED2, HIGH);
digitalWrite(LED6, HIGH);
digitalWrite(LED4, HIGH);
digitalWrite(LED5, HIGH);
}
if (input == 'x') {
digitalWrite(LED1, HIGH);
digitalWrite(LED3, HIGH);
digitalWrite(LED4, HIGH);
digitalWrite(LED6, HIGH);
digitalWrite(solenoide, HIGH);
}
if (input == 'y') {
digitalWrite(LED1, HIGH);
digitalWrite(LED6, HIGH);
digitalWrite(LED3, HIGH);
digitalWrite(LED4, HIGH);
digitalWrite(LED5, HIGH);
digitalWrite(solenoide, HIGH);
}
if (input == 'z') {
digitalWrite(LED1, HIGH);
digitalWrite(LED6, HIGH);
digitalWrite(LED3, HIGH);
digitalWrite(LED5, HIGH);
digitalWrite(solenoide, HIGH);
}
if (input == ',') {
digitalWrite(LED2, HIGH);
}
if (input == '.') {
digitalWrite(LED2, HIGH);
digitalWrite(LED6, HIGH);
digitalWrite(LED5, HIGH);
}
if (input == '!') {
digitalWrite(LED2, HIGH);
digitalWrite(LED3, HIGH);
digitalWrite(LED5, HIGH);
}
}
void setup() {
// Setup the pin modes
pinMode(solenoide, OUTPUT);
pinMode(motor, OUTPUT);
pinMode(LED1, OUTPUT);
pinMode(LED2, OUTPUT);
pinMode(LED3, OUTPUT);
pinMode(LED4, OUTPUT);
pinMode(LED5, OUTPUT);
pinMode(LED6, OUTPUT);
//When the buttons are pushed, digitalRead shows 0
pinMode(rotary_up, INPUT_PULLUP);
pinMode(rotary_down, INPUT_PULLUP);
pinMode(confirm, INPUT_PULLUP);
pinMode(back, INPUT_PULLUP);
//Initiate the serial monitor
Serial.begin(9600);
// attach the interrupt pin to a method
if (isInterrupt) {
attachInterrupt (digitalPinToInterrupt(confirm), ConfirmSwitchPressed, FALLING);
attachInterrupt (digitalPinToInterrupt(back), BackSwitchPressed, FALLING);
}
}
void loop() {
//Determining the number of messages/alerts and the length of each message
if (has_alert == true) {
string = alert[0];
message_length = string.length();
}
if (has_alert == false) {
string = INPUT_STRINGS[message];
message_length = string.length();
}
//Receiving the characters of incoming alerts
while (Serial.available()) {
//while there is data available on the serial monitor
//Serial.println(Serial.read());
incoming_alert += char(Serial.read()); //store string from serial command
}
insert_alert();
//Check if any movement has been made on the rotary encoder
if (millis() >= clock1) {
if (digitalRead(rotary_up) == LOW) {
if (start_letter + 12 <= message_length) {
start_letter += 12;
Serial.println("Next 12 letters");
}
}
if (digitalRead(rotary_down) == LOW) {
if (start_letter - 12 >= 0) {
start_letter -= 12;
Serial.println("Last 12 letters");
}
}
clock1 = millis() + INTERVAL1 ;
}
show_letters();
//For serial printing of demonstration
if (millis() >= clock2) {
if (incoming_alert != "") {
Serial.print("Incoming Alert: ");
Serial.println(incoming_alert);
}
//Serial.print("The current message or alert: ");
Serial.println(string);
clock2 = millis() + INTERVAL2 ;
}
//For 2-second duration of the vibration motor
if (millis() >= clock3 and vibration_on == true) {
digitalWrite(motor, LOW);
vibration_on = false;
Serial.println("vibration off.");
}
}
