For our Physical Computing class at Carnegie Mellon University, our final project was to prototype and design an assistive device for an older Osher student over a five-week period. Our Osher student was a lovely lady named Lynne who was looking for a way to rest her hands, which hurt due to arthritis in her fingers’ joints. Thus, we designed a large four-key keyboard to be used by her feet and aptly named it the Foot Keyboard. The keyboard can be plugged into a computer as arrow keys input or alternatively used as a standalone Simon Says game. The project began with interviewing Lynne in order to brainstorm about the device (Interview Documentation). We then received feedback on our prototypes of the device (Prototype Documentation). We concluded the project with a final critique from our Osher student, Lynne.
What we built:
Description
Our project is a foot arrow keyboard. There is a 14 in x 10 in platform with four arrow keys arranged in one row. The keys are styled to look and function like a gas pedal in a car. On the back edge of the platform is a hinge with a kickstand, which is adjustable to allow Lynne to set the keyboard at any desired height. There are two functions to the keyboard: a computer input mode and a Simon Says mode. For the computer input mode, Lynne can connect the keyboard to her laptop and browse the internet using the arrow keys, or she can play games that only require the four arrow keys (like Tetris or Snake). For the Simon Says mode, the keyboard is preprogrammed to randomly generate arrow key patterns for Lynne to repeat after. The difficulty of the Simon Says game starts out easy with a short pattern to repeat and gets more difficult as she continues to play.
Images:
CAD Images:
Overhead view of Foot Keyboard
Right side view of Foot Keyboard
Closer view of Power and Mode switches
Close up view of Left Arrow Key
Demonstration of joint movement for Arrow Keys and Kickstand:
TinkerCad Demo:
Narrative Sketch
After a long day at work, Lynne arrives home and considers how to spend the rest of her afternoon. She has spent the majority of the day using her hands and fingers on her work computer and the pain from her arthritis is very uncomfortable. After greeting Mendy and her two cats, she decides to give her hands a rest and occupy herself by using the Foot Keyboard! She powers on the device and plays a couple of rounds of Simon Says, to see if she can beat her record of 6 rounds from the previous day. She enjoys the fast-paced, challenging nature of the game.
A few rounds (and high scores) later, Lynne plugs the Foot Keyboard into her computer and switches the mode to ‘CPU Input’ Mode to play some online keyboard games, like Snake and Pac-man. After she feels like she has played enough games for the afternoon, she opens up her favorite online book and begins reading, using the Foot Keyboard to flip pages. When bedtime eventually rolls around, the pain in her hands has subsided, she unplugs her Foot Keyboard and prepares for bed.
How we got here:
There were multiple steps that we took to get to the final project after the initial design and prototyping such as multiple rewiring in TinkerCad, debugging the Code to work as intended, as well as changing up the design to create a believable, functional device for Lynne.
The TinkerCad wiring and the internal mechanic were made before the 3D Cad Model, which led to some mishaps as the button location didn’t match up with the model shown. Initially the ON/OFF and Mode button was a push button, but with some discussion with the team, we switched to a slide switch. In addition, before the code was added to the TinkerCad, the wiring for the push button was wired to work with a pull up input. We kept the pull up wiring and we decided to change the code to work with a pull up input. Once the design of the 3D Cad model was finished, we went back and changed the wiring to match up the location as close as possible. The Up and Down button location was swapped, as well as the indicated LED, in addition to the ON/OFF and MODE switch location. The wiring for the battery was also changed after creating the schematic, as I noticed that the ON/OFF button was not connected to the battery.
The very first draft of the TinkerCad wiring: The ON/OFF and MODE is push-button, and the arrow button is in a different order than the finalized Foot Keyboard.
For one section of the code: the computer input, we discovered that type of Arduino matters. While the code was simple with one add on of a library called “Keyboard.h” and input called “Keyboard.press” the code wasn’t able to be read with Arduino Uno. With research it was due to Uno’s different USB communication from other Arduinos. With further research Ardunio Leonardo, which has a certain USB communication that can read Keyboard and mouse inputs when connected to the computer, is the best option to use if we were to make it physically.
Arduino Uno Port: Computer Input Code not working/verifying with multiple code errors.
Arduino Leonardo Port: Verified Code after the port has been switched to Arduino Leonardo.
Another issue we ran into was integrating the code with the Tinkercad model. To represent variables with different “states”, such as the current Simon Says game mode or the direction of a button or LED, we decided to use enums in our code. This way, we could use a switch-case statement to cleanly execute different blocks of code depending on which state a variable was. We tried to use two enums (game_stage and dir) and this compiled mess-free in the Arduino IDE, but unfortunately caused mysterious bugs when pasting the code into the Tinkercad model. Unable to figure out the issue on our own, with the help of Zach, our professor, and Harshine, our TA, we were able to get the Tinkercad model working by replacing the second enum (dir) with the use of an integer (with values 0,1,2,3).
Tinkercad would throw compilation errors when attempting to use a second enum for direction.
Switched from an enum to using an integer to represent the four directions.
Conclusions and lessons learned:
We received a lot of helpful feedback and ideas from the final critique. One interesting idea which emerged from discussion was “Have you thought about adding a handle to the device?” We thought this was an excellent suggestion, since: 1) a handle wouldn’t be too hard to add to the design (a wide rectangular hole cut from base on the opposite side of the power/mode switches) and 2) it would make moving the keyboard (perhaps from the ground to the recliner) an easier endeavour. Other helpful suggestions appeared in the written feedback. One critiquer suggested mitigating typing fatigue by using “a nicer (mechanical) keyboard that isn’t as tiring to type on (since you don’t have to bottom the keys out)?” This suggestion ties into another provided during discussion, which was “There are large pre-made keys you can purchase which would allow you to avoid using foam.” Typing fatigue is definitely an important consideration, especially for a keyboard which works the typically unused foot and calf muscles. A feature to fine-tune during actual fabrication would be the thickness of the foam beneath the key-pedals to find a balance between sensitivity and fatigue. If this balance turns out to be difficult to obtain, looking into purchasing manufactured keys would definitely be a slightly more expensive, yet reasonable, suggestion to consider. A fourth suggestion, found in the written feedback, was to add a fifth “spacebar since a lot of games also require that button.” During the design phase, we played around with the idea of adding more keys but concluded that four was an optimal number, in terms of keeping the size fairly small and avoiding typing fatigue as much as possible.
Working remotely has been fairly strange as physical projects are more easily done in person. Overall, this experience was a positive one. Coordinating with each other online wasn’t too difficult. We found most difficulty in not being able to collaborate on unshareable files together, such as the Fusion 360 model and our Arduino code. Other than those challenges, we worked fairly cohesively, and there is not much we would do differently given the chance to work together again. Our client, Lynne, was extremely fun to work with, and she easily kept in contact with us as we worked on our device. We would’ve loved to meet with her in person and show her our work face to face, but chatting with her through Zoom has still been an enjoyable experience.
Our experience of working with an older individual was pleasant, as Lynne was very helpful in each of the steps we took, and how as a whole we were able to keep our conversation light and fun. What was difficult initially was coming up with a personalized device for Lynne from our first meeting, as many of our initial designs could have been used by the public. What helped us get through the initial challenge was, as mentioned before, how Lynne was easy to keep in contact with as we worked on our device. As we narrowed down our idea, we were able to ask additional questions, such as what games she enjoyed and what difficulties she may have with our initial foot keyboard design. Each critique we got from Lynne, we were able to slowly make the keyboard more personalized for her, which was a big help as we finalized our project. Because Lynne was tech-savvy there were no problems while working remotely, but if we were to do this again, during our initial interview, we could have asked multiple follow up questions, so we do not assume things, for example, when we initially designed it to be used on the floor rather than a recliner.
Technical details:
Design Files:
Final Project Lynne Laser Cut DXFs
TinkerCad:
TinkerCad Link: https://www.tinkercad.com/things/7FbfX3RFi6Y
TinkerCad: Foot Keyboard complete wiring
Schematic:
Foot Keyboard Final Schematic
Code:
/*
Project Title: Foot Keyboard - Team Lynne
Names: Sue Lee, Hojung Kim, Achilles Lin
Description: This code controls the two modes of the Foot Keyboard. The Simon Says mode randomly
generates sequences and asks the user to repeat the sequences in increments of 3 moves. The code
receives user input from the 4 momentary pushbuttons and delivers output through 4 LEDs and the
LCD screen. The Computer Input mode allows the user to use the 4 momentary pushbuttons to send
arrow key controls to a USB connected computer. The code continually loops to detect changes in
the mode switch, allowing the user to switch between modes at any point in either mode.
Pin mapping:
pin | mode | description
------|--------|------------
10 input Left Key Momentary Pushbutton
9 input Right Key Momentary Pushbutton
8 input Up Key Momentary Pushbutton
7 input Down Key Momentary Pushbutton
A0 output Left Key LED
A1 output Right Key LED
A2 output Up Key LED
A3 output Down Key LED
A4 output Mode Switch
Credit: Randomized numbers code from https://www.arduino.cc/reference/en/language/functions/random-numbers/random/
License Notice:
Copyright 2020 Sue Lee, Hojung Kim, Achilles Lin
Permission is hereby granted, free of charge, to any person obtaining
a copy of this software and associated documentation files (the "Software"),
to deal in the Software without restriction, including without limitation
the rights to use, copy, modify, merge, publish, distribute, sublicense,
and/or sell copies of the Software, and to permit persons to whom the
Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
*/
#include <LiquidCrystal.h>
#include <Keyboard.h>
LiquidCrystal lcd(12, 11, 5, 4, 3, 2);
#define LBTN_PIN 10
#define RBTN_PIN 9
#define UBTN_PIN 8
#define DBTN_PIN 7
#define LLED_PIN A0
#define RLED_PIN A1
#define ULED_PIN A2
#define DLED_PIN A3
#define MODE_PIN A4
bool prev_mode;
// Stages
enum game_stage {
OFF,
MODE_DISPLAY,
PRE_START,
ROUND_DISPLAY,
PATTERN_DISPLAY,
PATTERN_INPUT,
CORRECT_FEEDBACK,
GAMEOVER_FEEDBACK
};
/* tried to use enum but would crash tinkerCad
enum dir {
UP,
DOWN,
LEFT,
RIGHT
// {0,1,2,3} -> {UP,DOWN,LEFT,RIGHT} respectively
}; */
// must be multiple of 3
#define MAX_PTN_LEN 30
enum game_stage curr_stage = OFF;
enum game_stage prev_stage = OFF;
int round_num = 1;
int pattern[MAX_PTN_LEN] = { }; // initialize as zeros
int pattern_index = 0;
unsigned long stage_tmr = 0;
// instructions constants
#define MODE_DISPLAY_INTVL 2000
#define ROUND_DISPLAY_INTVL 2000
#define CORRECT_DISPLAY_INTVL 3000
#define GAMEOVER_DISPLAY_INTVL 3000
// in game constants
#define PTRN_DISPLAY_INTVL 2000
#define PTRN_DISPLAY_PADDING 500
#define PTRN_INPUT_INTVL 5000
/* HELPER FUNCTIONS */
// returns the amount of time that has elapsed since stage_tmr was last set
unsigned long elapsed() {
return millis() - stage_tmr;
}
// reset the stage_tmr with millis()
void reset_timer() {
stage_tmr = millis();
}
// returns true if the current game stage doesn't match the previous game stage
bool new_stage() {
return curr_stage != prev_stage;
}
// returns true if the current keyboard mode doesn't match the previous keyboard mode
bool new_mode() {
return digitalRead(MODE_PIN) != prev_mode;
}
// clears the LCD screen and prints str onto the LCD
void clearPrintLCD(String str) {
//insert LCD print code here
lcd.clear();
lcd.print(str);
}
// return the negated button state the desired key where
// {0,1,2,3} -> {UP,DOWN,LEFT,RIGHT} respectively
bool readBtn(int key) {
switch (key) {
case 0:
return !digitalRead(UBTN_PIN);
case 1:
return !digitalRead(DBTN_PIN);
case 2:
return !digitalRead(LBTN_PIN);
case 3:
return !digitalRead(RBTN_PIN);
}
return false;
}
// sets the LED output value of the desired key's LED where
// {0,1,2,3} -> {UP,DOWN,LEFT,RIGHT} respectively
void setLED(int key, bool output) {
switch (key) {
case 0:
digitalWrite(ULED_PIN, output);
break;
case 1:
digitalWrite(DLED_PIN, output);
break;
case 2:
digitalWrite(LLED_PIN, output);
break;
case 3:
digitalWrite(RLED_PIN, output);
break;
}
}
// sets the LED output values of all the key's LEDs
void setAllLEDs(bool output) {
for (int i = 0; i < 4; i++) {
setLED(i, output);
}
}
// initializes the simon says game round number and pattern sequence
void init_game() {
// reset round_num
round_num = 1;
// randomize pattern sequence
Serial.println("Generating sequence: ");
Serial.print("<");
for (int i = 0; i < MAX_PTN_LEN; i++) {
pattern[i] = random(4);
Serial.print(pattern[i]);
if (i < MAX_PTN_LEN - 1) {
Serial.print(",");
}
}
Serial.println(">");
}
void setup() {
// initialize inputs/outputs
Serial.begin(9600);
pinMode(UBTN_PIN, INPUT_PULLUP);
pinMode(DBTN_PIN, INPUT_PULLUP);
pinMode(LBTN_PIN, INPUT_PULLUP);
pinMode(RBTN_PIN, INPUT_PULLUP);
pinMode(ULED_PIN, OUTPUT);
pinMode(DLED_PIN, OUTPUT);
pinMode(LLED_PIN, OUTPUT);
pinMode(RLED_PIN, OUTPUT);
pinMode(MODE_PIN, INPUT);
lcd.begin(16, 2);
// if analog input pin 0 is unconnected, random analog
// noise will cause the call to randomSeed() to generate
// different seed numbers each time the sketch runs.
// randomSeed() will then shuffle the random function.
// Note: tinkerCad generates the same "random" sequence each time
randomSeed(analogRead(0));
// reads the keyboard mode and initliazes global mode variables
prev_mode = digitalRead(MODE_PIN);
}
void loop() {
// read keyboard mode
bool mode_switch_state = digitalRead(MODE_PIN);
if (new_mode()) {
if (mode_switch_state) {
// Just entered Simon Says game mode
// set game stage to MODE_DISPLAY
curr_stage = MODE_DISPLAY;
} else {
// Just entered Computer Input mode
prev_stage = OFF;
clearPrintLCD("CPU INPUT MODE");
Serial.println("Computer Input Mode");
}
}
bool ubtn_state;
bool dbtn_state;
bool lbtn_state;
bool rbtn_state;
// Check if keyboard is in Simon Says Game Mode
if (mode_switch_state) {
switch (curr_stage) {
// MODE_DISPLAY: print "SIMON SAYS MODE" to LCD for MODE_DISPLAY_INTVL
case MODE_DISPLAY:
if (new_stage()) {
// When entering MODE_DISPLAY stage, print "SIMON SAYS MODE" to LCD
prev_stage = curr_stage;
reset_timer();
clearPrintLCD("SIMON SAYS MODE");
Serial.println("MODE DISPLAY STAGE");
} else if (elapsed() > MODE_DISPLAY_INTVL) {
curr_stage = PRE_START;
}
break;
// PRE_START: print "PRESS -> TO START" until right button press (light up right LED with right button press)
case PRE_START:
if (new_stage()) {
// When entering PRE_START stage, print "PRESS -> TO START" to LCD
prev_stage = curr_stage;
reset_timer();
clearPrintLCD("PRESS -> TO START");
Serial.println("PRE-START STAGE");
}
// upon right key button press: light up right LED, intialize game
rbtn_state = readBtn(3);
if (rbtn_state) {
setLED(3, HIGH);
while (rbtn_state) {
rbtn_state = readBtn(3);
}
setLED(3, LOW);
curr_stage = ROUND_DISPLAY;
init_game();
}
break;
// ROUND_DISPLAY: print "ROUND <round_num>" for ROUND_DISPLAY_INTVL
case ROUND_DISPLAY:
if (new_stage()) {
// When entering ROUND_DISPLAY stage, print "ROUND <round_num>" to LCD
prev_stage = curr_stage;
reset_timer();
clearPrintLCD("ROUND ");
lcd.print(round_num);
Serial.println((String)"ROUND " + round_num);
} else if (elapsed() > ROUND_DISPLAY_INTVL) {
curr_stage = PATTERN_DISPLAY;
}
break;
// PATTERN_DISPLAY: print "SIMON SAYS..." to LCD and display <3x round_num> moves with LEDS: PTRN_DISPLAY_INTVL millis LED ON, PTRN_DISPLAY_PADDING millis LED OFF
case PATTERN_DISPLAY:
if (new_stage()) {
// When entering PATTERN_DISPLAY stage: print "SIMON SAYS..." to LCD, reset pattern_index
prev_stage = curr_stage;
reset_timer();
clearPrintLCD("SIMON SAYS...");
pattern_index = 1;
Serial.println("PATTERN_DISPLAY MODE");
}
// light up each LED corresponding to sequence
if (pattern_index > (round_num * 3)) {
// once all moves have been displayed for current round, move to next stage
curr_stage = PATTERN_INPUT;
} else if (elapsed() < PTRN_DISPLAY_INTVL) {
// turn on LED
setLED(pattern[pattern_index - 1], HIGH);
Serial.println((String)"Pattern index: " + pattern_index + ", Key: " + pattern[pattern_index - 1]);
delay(PTRN_DISPLAY_INTVL);
} else if (elapsed() > PTRN_DISPLAY_INTVL) {
// turn off LED
Serial.println("Turn LED OFF");
setLED(pattern[pattern_index - 1], LOW);
pattern_index++;
delay(PTRN_DISPLAY_PADDING);
reset_timer();
}
break;
// PATTERN_INPUT: print "YOUR TURN!" to LCD and accept user input of pattern, light up LEDs with button presses
case PATTERN_INPUT:
if (new_stage()) {
// When entering PATTERN_DISPLAY stage: print "SIMON SAYS..." to LCD, reset pattern_index
prev_stage = curr_stage;
reset_timer();
clearPrintLCD("YOUR TURN");
pattern_index = 1;
}
if (pattern_index > (round_num * 3)) {
// once all moves have been inputted for current round, move to next stage
curr_stage = CORRECT_FEEDBACK;
} else if (elapsed() > PTRN_INPUT_INTVL) {
// game over if time runs out
curr_stage = GAMEOVER_FEEDBACK;
} else {
ubtn_state = readBtn(0);
dbtn_state = readBtn(1);
lbtn_state = readBtn(2);
rbtn_state = readBtn(3);
int curr_dir = pattern[pattern_index - 1];
int input_dir = 0;
// check if any button has been pressed
if (ubtn_state || dbtn_state || lbtn_state || rbtn_state) {
if (ubtn_state) {
Serial.println("UP");
input_dir = 0;
} else if (dbtn_state) {
Serial.println("DOWN");
input_dir = 1;
} else if (lbtn_state) {
Serial.println("LEFT");
input_dir = 2;
} else if (rbtn_state) {
Serial.println("RIGHT");
input_dir = 3;
}
// turn on corresponding key LED until button unpressed
setLED(input_dir, HIGH);
int pressed_btn_state = readBtn(input_dir);
while (pressed_btn_state) {
pressed_btn_state = readBtn(input_dir);
}
// turn off LED
setLED(input_dir, LOW);
// check if correct key was pressed
if (curr_dir == input_dir) {
// correct key pressed
pattern_index++;
reset_timer();
} else {
// incorrect key pressed
curr_stage = GAMEOVER_FEEDBACK;
}
}
}
break;
// CORRECT_FEEDBACK: print "NICE JOB!" for CORRECT_DISPLAY_INTVL
case CORRECT_FEEDBACK:
// print "NICE JOB!" for CORRECT_DISPLAY_INTVL
if (new_stage()) {
prev_stage = curr_stage;
reset_timer();
clearPrintLCD("NICE JOB!");
Serial.println("CORRECT FEEDBACK STAGE");
} else if (elapsed() > CORRECT_DISPLAY_INTVL) {
curr_stage = ROUND_DISPLAY;
round_num++;
}
break;
// GAMEOVER_FEEDBACK: print "GAME OVER :(" and flash all LEDs on for GAMEOVER_DISPLAY_INTVL
case GAMEOVER_FEEDBACK:
if (new_stage()) {
prev_stage = curr_stage;
reset_timer();
clearPrintLCD("GAME OVER :(");
setAllLEDs(HIGH);
Serial.println("GAMEOVER FEEDBACK STAGE");
} else if (elapsed() > GAMEOVER_DISPLAY_INTVL) {
// turn off all LEDs
curr_stage = PRE_START;
setAllLEDs(LOW);
}
break;
// default handles OFF stage; curr_stage should never be OFF
default:
clearPrintLCD("ERROR");
Serial.println("ERROR");
delay(500);
}
} else { // computer input mode
Keyboard.begin(); //begin keyboard
//Up Button
if (readBtn(0)) {
Keyboard.press(KEY_UP_ARROW);
Serial.println("KEY_UP_ARROW PRESSED");
}
else {
Keyboard.releaseAll();
}
//Down Button
if (readBtn(1)) {
Keyboard.press(KEY_DOWN_ARROW);
Serial.println("KEY_DOWN_ARROW PRESSED");
}
else {
Keyboard.releaseAll();
}
//Left Button
if (readBtn(2)) {
Keyboard.press(KEY_LEFT_ARROW);
Serial.println("KEY_LEFT_ARROW PRESSED");
}
else {
Keyboard.releaseAll();
}
//Right Button
if (readBtn(3)) {
Keyboard.press(KEY_RIGHT_ARROW);
Serial.println("KEY_RIGHT_ARROW PRESSED");
}
else {
Keyboard.releaseAll();
}
delay(200);
Keyboard.end(); //stops keybord
}
prev_mode = mode_switch_state;
delay(50);
}
Comments are closed.