For this project, our team was tasked with creating an assistive device for our Osher student Cynthia. The goal was to design a piece of technology to help her in her daily life. We conducted an interview with Cynthia to gain better insight into her life and identify potential problems we could address with our project. Our team decided to design an automated dog trainer to help train Cynthia’s dog Zimi to stop chewing on her sandals and other valuables. We documented our prototypes before moving onto designing our final device.

What We Built

Our device is a dog trainer that senses the presence of an object placed on top of it. If the device senses the object has been moved, alerting that Zimi is chewing on the object, it will trigger a high pitched alarm sound as punishment. The sound is at a frequency that dogs can hear, so as to not disturb Cynthia. There is an additional switch that can turn the alarm on and off to allow Cynthia to remove the object without sounding off the alarm. Other features include a blue light that turns on to indicate the alarm going off, as well as a separate alarm that goes off at 8am to remind Cynthia to take Zimi outside. Over time, the goal of the device is to train Cynthia’s dog to learn what items he should not be chewing  on.

FINAL RenderingS

Labeled components of our device modeled with Fusion 360 .

Overall view of the device: A circular, elevated platform with a wooden base and black glass for the paw print detail.

Detailed view of the device components. The keypad is used to input time and alarm sound settings. The LCD screen displays device modes and user inputs.

Battery door located at bottom of device that can be unscrewed to replace battery power.

Device measurements for scale (mm)

PIR sensors located beneath the black glass. There is a lower surface panel to hold components.

Side view of the device.

TinkerCad INTERACTION

Narrative Sketch

Cynthia is in her kitchen working on her book. Meanwhile, Zimi is walking around the house looking for something to chew on. She sees paper on the platform, her new favorite object to chew on. Zimi walks over and grabs the paper. Suddenly, Zimi hears dog whistles coming from the device. She runs away and the whistling stops. Zimi tries again and the same sound goes off. She gives up and looks for other things to do.

The next morning, Cynthia is eating her breakfast. At 8AM, she hears the device going off and sees the LCD message reminding her to walk Zimi. It was too loud of a sound in the morning, so she turns down the volume. She turns off the alarm, finishes her breakfast, and brings Zimi outside.

In the afternoon, Cynthia resumes in editing her book. One of her papers fell to the ground without her realizing it. Zimi sees the paper, but decides not to bother with it.

How We Got Here

The process of creating this device was definitely a learning experience for us as we had to overcome various challenges along the way. The initial sketch for the device gave us a solid starting point to work off of, but we later made significant changes to the design for both aesthetic and functional purposes. We cycled through iterations of the device, adding new components such as an 8am alarm and volume adjustment after receiving feedback from Cynthia. In our work plan, we scheduled to front load most of the work, which meant finishing the circuits and model renderings within a week. However, we ended up diverging from our original schedule due to last minute revisions. 

Initial concept sketches.

One major setback was that TinkerCad doesn’t have the Real Time Clock part, so we had to improvise by creating an alternate means of tracking time through a keypad as the user interface to set the time and then having the device calculate current time from that input. As a result, we got rid of the potentiometer which we initially planned to have control volume since we could just incorporate volume settings onto the keypad. 

These changes to the circuit also meant changes to the physical model design. This posed another challenge since we did not anticipate these changes and had to consider ways to add this new keypad component which was quite large and clunky in appearance. We considered adding a separate platform to fit the keypad, but it was not as clean design-wise. In the end, we resized and rearranged the new components to all fit on the top surface which ended up looking alright. 

Added potentiometer to adjust alarm volume which was later replaced with keypad .

Rendering the models was time-consuming due to the fact that the renders were made too soon without accounting for last minute changes which required re-rendering. In addition, the quality of the renderings were turning out blurry, and it was only until much later that we discovered it was because the image dimensions were set to very small values, making them pixelated. After increasing image size to 3840px by 2160px, the quality improved dramatically, and we were finally able to get decent renderings of the finished device.

Exported cloud renderings which had to be re-rendered to match updated design.

Blurry rendering of unfinished model.

Render settings menu with the default image width and height set to a small pixel size.

Conclusion

We received a mix of feedback during the presentation of our final device. Overall, people liked the aesthetics of our project and the concept behind it. However, some expressed concerns if this form of positive punishment will be successful in real life, which was also a concern we had while developing the device. We mainly worked based on a theory and ran with it, but understood that such a device would have to be tested on Zimi to see if it will actually be useful, though such testing would take a while to get any conclusive results since our device hopes to affect behavior. Others had more specific concerns such as:

Is the alarm for Cynthia with the unit? Would she be able to hear it if it is placed on top of her shoe?

Would this only work for 1 pair of shoes? Would Cynthia need multiple of the devices? For all her shoes?

In regards to the audibility issue, we positioned the speaker away from the middle of the device and in the corner with all of the other electronic inputs/outputs such as the keypad and LED. The device was built large enough that this area would not be covered by a shoe for example. Additionally, Cynthia wanted the ability to change the volume of the device so she can increase/decrease the volume depending on what room she places the device in so she can reasonably hear it. But, we agreed that we can continue to optimize the positioning of the speaker on the device and even the type of speaker. A lot of the feedback was concerned on the practicality of the device’s application with a shoe. We used shoes as an example of what Zimi likes to chew on, but we hope that this device has the versatility to train Zimi not to chew on a variety of small objects. Ideally, we hope Zimi will associate the objects placed on the device with punishment and bad things she should not be chewing on.

After our first prototype, Cynthia expressed Zimi chewing on shoes was no longer a problem since Cynthia decided to hide all of her shoes. However, we agreed hiding shoes might not be practical in the long run as we wanted to teach and train Zimi as a more long-lasting solution to her chewing habits. Cynthia expressed concerns that Zimi is starting to chew on paper she finds on the floor. If our device was built, we would first try it with paper and see how her behavior changes. If successful and Zimi finds other things to chew, then we would potentially put whatever new object Zimi likes to chew onto the device so that we can prime her not to chew on those types of items.

Collaborating and developing a device remotely had its challenges. Due to online learning, we shifted to using mainly CAD and TinkerCad to develop and present our work. A lot of the project work was split into individual tasks which we worked separately on. Some tasks were more difficult to collaborate on remotely, for instance creating the CAD model or wiring the circuit. This task allocation actually turned out to work pretty well since we were able to concentrate on our own side of the project. In the end, although we each addressed separate project domains, we were able to create a cohesive device by combining our components.

Reflecting on the process of building this device, one of the biggest challenges was using TinkerCad, which is slow, has limited parts, and definitely buggy. For instance, we originally wanted to use an IR remote instead of a keypad. However, the library for the IR remote could not be added to the code without creating errors. Additionally, simulating the interaction is not the same as real life. The project would have definitely been easier and more constructive if we have the electronic parts with us to physically troubleshoot challenges in the wiring and code. If we were to do the project again, we would like to do this part of the project physically in real life, so that we can have a better understanding of the functionality of the device and better improve it.

The CAD portion also had it has its own challenges, but in the end improved the quality of the overall project. For future CAD models, we learned it’s important to leave room for error and design adjustment before moving onto final rendering. We would have had to use CAD anyway, but since we didn’t have to fabricate it, it expanded the potential of the overall physical device. Our specific device can be fabricated, but the situation allowed us to design more complex designs without being held back by fabrication difficulties.

Working with an older person and designing a device specifically for them was a new experience for us. We thought it was a wonderful experience learning about Cynthia during our initial interview with her. She has lived in Pittsburgh for a long time so she was able to share some insights about her favorite places to eat and drink. One of the major takeaways that became apparently clear especially after the final presentations were that many of the devices built address issues that many people people might have problems with and have the potential to reach a wider population.

Technical Details

TINKERCAD

TinkerCAD (https://www.tinkercad.com/things/91w05kUAJCs)

Breadboard image of Tinkercad model

Code

#include <Adafruit_NeoPixel.h>
#include <Keypad.h>
#include <LiquidCrystal.h>

/** 
   Title: Dog Trainer
   Author: Gracia Genero
   Description: This code controls the components listed below. 
   It fires Zimi's alarm through the speaker/LED when motion is detected
   through the PIR Sensor. The PIR sensor and Zimi's alarm can be deactivated
   by flipping the switch. Additionally, this device keeps track of time'
   time, but requires the user to input a current military time to do so accurately. 
   At 8AM a separate alarm goes off to remind the user (Cynthia) to walk her dog
   Zimi. This alarm can be turned off through the keypad input. The volume of 
   Cynthia's alarm can also be adjusted through the keypad. The LCD screen displays messages
   when the alarm is activated or when an input is in progress of being typed into the
   keypad. Otherwise, it displays the calculated current time and volume. 
   
  
   Pin mapping:
     
   pin        | mode   | description
  ------------|--------|------------------------------------
   A3         | input  | PIR motion sensor
   A4         | output | NeoPixel Ring 16
   A5         | output | speaker
   A2         | input  | switch
   A0,A1,11,10| input  | keypad - row pins
   8,7,6,5    | input  | keypad - column pins
   4,3,9,2    | input  | LCD - DB4-DB7
   13         | input  | LCD - register select pin
   12         | input  | LCD - enable pin
   

  Credits:
  https://www.tinkercad.com/things/ascn1ro2gFR-lcd-display
  https://www.tinkercad.com/things/jLFSyDIIEpj-pir-motion-sensor-with-arduino-blocks
  https://www.tinkercad.com/things/k4BAT8c26NN-print-4x4-keypad-input-to-lcd
 
  
 **/

const int SPEAKERPIN = A5;
const int SWITCHPIN = A2;
const int LEDPIN = A4;
const int LEDCOUNT = 16;       // number of NeoPixels in series
const int PIRPIN = A3; 
int pirState = LOW;             // we start, assuming no motion detected
int pirVal = 0;                 // variable for reading the pin status
int ledState = LOW;
int piezState = LOW;
String vol = "5"; // range 0-9
String setTime = "0500";
String keypadInput = "";
String alarmTime = "0800";
String currentTime = "0500";
int CynthiaAlarm = 0;
int turnedOff = 0;

int editMode = 0; 
int volMode = 0;
unsigned long int setMillis= 0; 
int LCDCol = 0;

const byte ROWS = 4; //four rows
const byte COLS = 4; //three columns
char keys[ROWS][COLS] = {
  {'1','2','3','A'},
  {'4','5','6','B'},
  {'7','8','9','C'},
  {'*','0','#','D'}
};
byte rowPins[ROWS] = {A0, A1, 11, 10}; //connect to the row pinouts of the keypad
byte colPins[COLS] = {8, 7, 6, 5}; //connect to the column pinouts of the keypad
Keypad keypad = Keypad( makeKeymap(keys), rowPins, colPins, ROWS, COLS );

// Object
Adafruit_NeoPixel rgb(LEDCOUNT, LEDPIN, NEO_RGB + NEO_KHZ800);
LiquidCrystal lcd(13, 12, 4, 3, 9, 2);



void setup() {
  pinMode(SWITCHPIN, INPUT);
  pinMode(LEDPIN, OUTPUT);      // declare LED as output
  pinMode(PIRPIN, INPUT);     // declare sensor as input
  pinMode(SPEAKERPIN, OUTPUT);
  
  lcd.begin(16, 2);
  Serial.begin(9600);

}

void loop(){
  
  char key = keypad.getKey();
  if (key){
    Serial.println(key);
    
    if (key == '*') { 
    setTime = keypadInput; 
    setMillis = millis(); 
    Serial.println("Set time = " + String(setTime) + " at millis "+ String(setMillis));
    editMode = 0;
    keypadInput = "";
    Serial.println("Time Set");
    LCDCol = 0; 
    lcd.clear();
    }
    
    if (editMode == 1) {
      keypadInput += String(key);
      lcd.setCursor(LCDCol, 1);
      lcd.print(key);
      LCDCol += 1;
    }

    if (volMode == 1) { 
      vol = String(key);
      volMode = 0; 
       }
    
    if (key == 'A') { 
      editMode = 1;
      Serial.println("Setting Current Time");
      lcd.clear();
      lcd.setCursor(0,0);
      lcd.print("Set current time");
 
    }
    
    if (key == 'B') { 
      volMode = 1;
      Serial.println("Changing Volume");
      lcd.clear();
      lcd.setCursor(0,0);
      lcd.print("Set volume");
    }
  }
  
  Serial.println("Volume = " + vol);
  
  // Calculates current time based on last current time by the user
  int deltaMin = ((millis() - setMillis) / 60000) % 60;
  int deltaHour = ((millis() - setMillis) / 60000) / 60; 
  
  String setMin = setTime.substring(2);
  int min = setMin.toInt();
  String setHour = setTime.substring(0,2); 
  int hour = setHour.toInt();
  
  int newMin = (min + deltaMin) % 60;
  String newMinStr = String(newMin);
  if (((newMin % 10) >= 0) and ((newMin / 10) < 1)) {
  	newMinStr = "0" + newMinStr; 
  }
  if (((min + deltaMin) / 60) >= 1) { deltaHour += 1; }
  int newHr = (hour + deltaHour) % 24;
  String newHrStr = String(newHr);
  if (((newHr % 10) >= 0) and ((newHr / 10) < 1)) {
  	newHrStr = "0" + newHrStr; 
  }
  currentTime = newHrStr + newMinStr;
  Serial.println("SetTime = " + setTime + ", Current Time = " + currentTime);
  
  if (((editMode == 0) and (volMode == 0)) and (CynthiaAlarm == 0)) {
      lcd.setCursor(0, 0);
      lcd.print("Time: " + currentTime);
      lcd.setCursor(0, 1);
      lcd.print("Volume: " + vol);
    }
  
  int switchState = digitalRead(SWITCHPIN);
  pirVal = digitalRead(PIRPIN);  // read input value
  Serial.println("PIRSensor = " + String(pirVal));
  
  if ((currentTime == alarmTime) and (turnedOff == 0)) {
    CynthiaAlarm = 1; 
    }
  if (currentTime.toInt() >= 803) {
    CynthiaAlarm = 0;
    turnedOff = 0;
    lcd.clear();
  }
  if ((key == 'C') and (currentTime.toInt() <= 803) and (currentTime.toInt() >= 800)) {
  	turnedOff = 1;
    CynthiaAlarm = 0;
    lcd.clear();
  }
  if (CynthiaAlarm == 1) {
      tone(SPEAKERPIN, 100);
      lcd.clear();
      lcd.print("It's time for");
      lcd.setCursor(0,1);
      lcd.print("a walk!"); 
  }
  else {
  if (switchState == LOW) {
  if (pirVal == HIGH) {          // check if the input is HIGH
    // turn on LED blue
    for(int i=0; i<rgb.numPixels(); i++) {
    rgb.setPixelColor(i, 0, 0, 150);    // sets the RGB values for pixel number i  
    // rgb.show();
    }  
    rgb.show();
    analogWrite(SPEAKERPIN, vol.toInt()*14); // sets duty cycle or volume of piezo buzzer 
    tone(SPEAKERPIN, 25000); // speaker sends off sound frequencies only dogs can hear
    if (pirState == LOW) {
      Serial.println("Motion detected");
      pirState = HIGH;
    }
  } 
  else {
    noTone(SPEAKERPIN); // turns off speaker
    rgb.clear(); // turn off LED
    rgb.show(); 
    if (pirState == HIGH){
      Serial.println("Motion ended");
      pirState = LOW;
    }
  
  }
  }
    
  else {
  
  noTone(SPEAKERPIN);
  rgb.clear(); // turn off LED
  rgb.show(); 
  }
  }
}

SCHEMATIC

Schematic of dog trainer device

Design Files

Dog Trainer Design Files (Fusion 360)

For the final project of Physical Computing(60-223) course at Carnegie Mellon University, each group of three CMU students was assigned to an Osher student to build an assistive device for the older adults. Our group had the fortune to work with an amazing Osher student who herself is a CMU alum, Emily Eckel. Through multiple stages of interviews, ideations, discussion, and critiques, we decided to create an entertaining ‘Mouse Detector’ that can detect mouse at her house so that she can bring her son’s cat from upstairs. The two most important goals of this project were functionality and entertainment. The device should be able to detect a mouse, nothing else, with high accuracy, and be able to notify Emily. On top of that, due to the COVID-19 pandemic situation, Emily had to spend more time at her house which makes her a bit depressed; thus she wanted a device that can give her small joy in daily life.

Although this project was initially planned to have an actual physical prototype in the end, because of the COVID-19 pandemic situation where all CMU courses have officially conducted remotely, we had no choice but to make this project virtual. To provide a more constructive and tangible overview of our idea to Emily and everyone in this process, we decided to make a physical crafted prototype, a CAD design simulation, and a circuit/technical prototype. For your reference, more details about the initial ideation and plans can be found here, and the initial model and details can be found here.

What We Built

We built a mouse detector, with two parts: the sensor unit which detects if a mouse is nearby, and the control panel unit that notifies Emily if a mouse has been detected and allows her to control the notifications. When a mouse is detected, the control panel lights up, and a small door opens and a mouse sign emerges. If it is daytime, music from Mickey Mouse plays. The volume knob controls the loudness of the music, and the brightness knob controls the brightness of the panel when it lights up. When the reset button is pressed, the mouse sign goes back in and the door closes, and light and music turns off.

Final appearance rendering

Overall image (character panel going back in after device has been reset)

Volume and brightness control knobs

Speakers

Notification on

Tinkercad Interaction

Physical MOCKUP

A front view of the controller unit crafted prototype (two green and red knobs each are volume and brightness controller, a door and a mouse panel is a cuckoo bird clock-like notification, and a yellow rectangular button is a reset button)

A top view of the controller unit crafted prototype (two yellow circles are a speaker unit for notifications)

A side view of the controller unit crafted prototype

A back view of the controller unit crafted prototype (magnets for the convenience of installing it on the fridge)

A video of how the cuckoo bird clock-like notification works like

A front view of the sensor unit crafted prototype (cheese shape for entertainment)

A back view of the sensor unit crafted prototype (a plug is directed connected to the unit)

A side view of the sensor unit crafted prototype (blue and red pins represent IR proximity sensors that will be installed toward the fridge)

How the controller unit would look like when it is installed on the fridge

How the sensor unit looks like when it is plugged into an outlet (sensors are facing toward the fridge where Emily said mouse appear the most at her house)

NARRATIVE SKETCH

While Emily is at home, a mouse appears from behind the fridge, as usual. As soon as the detector that is plugged into the outlet by the fridge detects the moving mouse, the control panel above on the side of the fridge glows. The fun music plays, and the mouse character pops out of the small doors on the control panel. The sound wouldn’t go off at night, but since it’s daytime and Emily is awake, she needs to come to the fridge and press the reset button to stop the notification. She then calls her son to come into the house and for his cat to catch the mouse as soon as possible.

How We Got Here

Our original schedule

When we were finalizing mechanism methods and components, we took some more time than we had planned in our work plan. This was because when we made the original schedule, we did not anticipate making such a big change in the final product. Although we spent more time in brainstorming and revising out design, the schedule worked out smoothly in the end, it took us less time to build the actual prototypes because the discussion we had earlier was very detailed.

Our prototype was a single-component mouse detector near the ground, with both the detection, control and notification functionalities. However, although a single-component device might be easy to use, we realized having both controller knobs and reset button near the floor would be inconvenient and dangerous for Emily whose back hurts when she bends her back and stands up. On top of that, we found it more visible to have notifications on our eye level rather than having it on the floor, and especially near the fridge where Emily consistently passes by and uses. After our in-depth discussion about the project, we decided to have two components—a sensor unit near the floor and a controller unit on the fridge—in our final product. We first took some time to figure out how they would be powered, and also how they would be connected to each other. Later we essentially redesigned our 3D models, and later built a physical mockup and a corresponding Tinkercad circuit.

A front overview of the digital rendering draft

A back overview of the digital rendering draft

For designing the appearance and creating digital renderings, the main components within the control panel unit were kept as they were developed early on, including 2 speakers, 2 knobs, a pair of doors, 1 character panel,  1 reset button, and magnets on the back side. Since the very first iterations, the form was modified to give a soft and fun look, and knobs were made taller for better usability. The number of magnets was increased for them to be distributed more evenly and for sturdiness.

A front overview of the digital rendering draft with material details added

Then, we changed from a simple opaque material to a yellow, semi-transparent material to make it seem less serene or too much like a common house appliance, and more fun, as well as being able to transmit light when the notification light turns on. Towards the end, we learned that it actually takes a very long time to render transparent material, so we had some difficulty in producing high-quality light-on renders. We could have allocated more time to make this part go more smoothly.

We also used metal to clearly distinguish knobs to be interacted with and the bottom unit, which would contain any internal parts. We also added texture (texts layered on the front surface) to indicate what the knobs are and how to use them.

Electronically, we decided to power the detector part by plugging in to the outlet, and the control panel part by a USB cable from the outlet, after the hearing from Emily that she wanted them to powered by wall power. The two parts would wirelessly communicate through radio transceivers attached to the two Arduinos. However, since Tinkercad does not have the radio transceiver part, in our Tinkercad simulation the two Arduinos are directly connected by wires with two I/O pins, since the communication was simple enough. Another change from our original prototype was changing the sensor component from a PIR sensor to an IR proximity sensor, after feedbacks from our instructor Zach. Since Tinkercad does not have the part, we used an ambient light sensor to model the IR proximity sensor component.

The interior of the sensor unit. There are multiple head pins which the end are hot-glued

For crafting, the hardest part was deciding which materials to use. Because we  are not testing the whether the device actually works, the most important aspect of these materials were how easy it is to craft, and how close can it reflect our design. Due to the pandemic, there were some limitations in getting materials. We used corrugated cardboards for the general structure, stamp-looking fridge magnets for controller knobs, and two different colors of plastic head push pins which each represents IR receiver and IR emitter LED. However, using push pins were quite dangerous because the end parts are very sharp. For safety reasons only, we needed to hot-glue the end part although these details were not necessary.

The connection between the appropriate size plug and the sensor unit

On top of that, there were some difficulties finding an appropriate size and weight of a plug connected to the sensor unit. If we had a chance to implement our design in the lab, this would not have be an issue; however, as we used a cardboard to make the body structure of the sensor unit, when the plug was too big and heavy, the structure collapsed. After multiple candidates, we found the plug that fits the sensor unit size and also not too heavy so that the craft can maintain its shape.

4. Conclusions and lessons Learned

During the final critique, we got multiple positive feedbacks about dividing the device into two parts, which was very rewarding because brainstorming and refining the idea was one of the biggest challenges in our project.

However, some feedback raised the concern about how accurate the sensor unit will be. Given the current situation, it is difficult to physically test, but we could have done some more research to predict better what the optimal angle or area size would be for the specific type and number of detectors. Moreover, in the normal lab setting, it would be hard to test it actually with mouse present; thus, if we were to test it in the lab, we would need to come up with a valid alternative testing method.

One feedback we got was that the novelty and entertainment components of this product might wear out. We think that this is definitely a valid feedback, but we think adding the fun part (the “cuckoo clock” notification) makes designing the product a bit more interesting compared to a simpler design. If we had more time, adding a function to change the alarm sound would make interacting with the product more interesting, and avoids the possibility that the user get annoyed by the alarm sound after using it for a while.

On top of that, during the feedback session with the final crafted prototype, there was a feedback that it would be better to have an on/off switch and a switch that can manually set the day and night mode. We had a discussion about adding such features, but ended up concluding that it is more important to make the device more concise and small as Emily requested it to be than to add additional features. However, there was a feedback during the final presentation suggesting an idea or merging volume and brightness knobs into one, which we have not thought of but seems very interesting and doable. If we have further opportunities to develop this device, we would like to further discuss about this point.

Regarding the teamwork, because our team members were working remotely from each other in different countries, it was difficult to share progress and iterations, and communication was often rather simplified. This made it a little harder to put together ideas and developments, so we decided that it would be the best if each person having a focus, and the process of discussing as a group first, building our own parts/prototypes, discussing our individual decisions and potential changes, and revising our parts. This worked fairly well as we could each take an ample amount of time to develop diverse prototypes, and take good feedback from each other.

Working with a client itself was definitely new to us but very valuable. Unlike following instructions or making a device for myself which were two prior projects conducted during the course, making a device for a person that I have never met before taught me how to interact with clients and how to get to know the client personally. Most importantly, we realized there were many considerations and assumptions we made without thorough decision making processes when we were doing a project on ourselves. For example, it would have been obvious to make the device installed on the desk without any question even just by imagining how we would use it, because that is where we go often; but it was not for Emily. We learned how to interview clients and to what extent we should be interviewing the client in terms of details, and in many cases interviewing the client life pattern turns out to be useful although it does not seem to be related to the project in the beginning.

We were so fortunate to have Emily, who is not only so warm and nice but also was very engaged in project such by giving objective feedbacks and her own ideas, as our client. She was also so opened in giving many small details about her life in general, which helped us a lot in constructively building the model. It truly made the virtual process a lot easier and more productive. However, we could have met her a couple more times to ask her for feedback and take some opportunities to gain more insights, reactions, and any recommendations. During this process, we also learned a little bit about what are some things to consider when designing for older people, such as the terms we use to describe their actions or needs, or keeping in mind their physical capabilities and limitations.

Technical Details

Tinkercad Link

Link

Breadboard Image

Code for the sensor unit’s Arduino

/*
 * Final Project: Mouse Detector for Emily
 * Youie Cho, Yun Lee, Alan Zhu
 *
 * Description: This is the code that would be on the sensor unit's
 * Arduino. Essentially it reads an input of a IR Proximity sensor
 * (modeled by a phototransistor in Tinkercad) to check if a mouse
 * is nearby. If a mouse is detected, it sends a digital signal to
 * the control unit's Arduino. It also reads a photoresistor to
 * identify if it is daytime, and sends the digial signal to the
 * control unit.
 * 
 * Pin mapping:
 * pin   | mode   | description
 * ------|--------|------------
 * A0     input    phototransistor (modeling a IR Proximity sensor)
 * A1     input    photoresistor
 * 12     output   direct digital output to the control unit  
 * 13     output   direct digital output to the control unit
 * 
 */

//input pins
const int PHOTOTRANSISTOR_PIN = A1; 
const int PHOTORESISTOR_PIN = A0; 

//output pins, both send signals to the control panel arduino
const int MOUSE_PIN = 13;
const int DAYTIME_PIN = 12;

int lightLimit = 650;
int mouseLimit = 900;
int lightVal, mouseVal;

void setup()
{
  Serial.begin(9600);
  pinMode(PHOTOTRANSISTOR_PIN, INPUT);
  pinMode(PHOTORESISTOR_PIN, INPUT);
  pinMode(MOUSE_PIN, OUTPUT);
  pinMode(DAYTIME_PIN, OUTPUT);
}

void loop()
{
  //Reads photoresistor, if the readings exceed the preset limit,
  //it is daytime. Sends corresponding digital signal to the control
  //panel Arduino.
  lightVal = analogRead(PHOTORESISTOR_PIN); 
  if(lightVal >= lightLimit) digitalWrite(DAYTIME_PIN, HIGH);
  else digitalWrite(DAYTIME_PIN, LOW);
  
  //Reads phototransistor value (modeling IR Proximity Sensor),
  //if exceeds preset limit, mouse is detected. Sends corresponding
  //digital signal to the control panel Arduino.
  mouseVal = analogRead(PHOTOTRANSISTOR_PIN); 
  if(mouseVal >= mouseLimit) digitalWrite(MOUSE_PIN, HIGH);
  else digitalWrite(MOUSE_PIN, LOW);
  
  delay(150);
}

Code for the control panel unit’s Arduino

/*
 * Final Project: Mouse Detector for Emily
 * Youie Cho, Yun Lee, Alan Zhu
 *
 * Description: This is the code that would be on the control panel
 * unit's Arduino. It reads the inputs from the sensor unit's Arduino
 * of whether a mouse has been detected and if it is daytime. If a mouse
 * is detected, the LED lights up, two servomotors rotate to open a
 * door. The speaker plays music in addition if it is daytime. The brightness
 * knob potentiometer sets the brightness level for the LED, and the volume
 * knob sets the loudness of the speaker. If the reset button is pressed,
 * the LED turns off, two servomotors return to its original position and
 * closes the door, and the music is turned off.
 * 
 * Pin mapping:
 * pin   | mode   | description
 * ------|--------|------------
 * A1     input    potentiometer (brightness knob)
 * A3     output   LED
 * A4     output   servo 1  
 * A5     output   servo 2
 * 3	  input    reset button
 * 7      output   piezo speaker
 * 12     input    direct digital input from the sensor unit Arduino
 * 13     input    direct digital input from the sensor unit Arduino
 *
 */

#include <Servo.h>

Servo servo1, servo2;

//input pins
const int MOUSE_PIN = 13; // from the other Arduino
const int DAYTIME_PIN = 12; // from the other Arduino
const int LIGHT_POT_PIN = A1;
const int BUTTON_PIN = 3;

//output pins
const int PIEZO_PIN = 7;
const int LED_PIN = A3;
const int SERVO1_PIN = A4;
const int SERVO2_PIN = A5;

int photoLimit, photoVal, lightVal;
bool dayTime, mouse, button;

void setup()
{
  Serial.begin(9600);
  servo1.attach(SERVO1_PIN);
  servo1.write(0);
  servo2.attach(SERVO2_PIN);
  servo2.write(180);
  pinMode(MOUSE_PIN, INPUT);
  pinMode(DAYTIME_PIN, INPUT);
  pinMode(LIGHT_POT_PIN, INPUT);
  pinMode(BUTTON_PIN, INPUT_PULLUP);
  pinMode(PIEZO_PIN, OUTPUT);
  pinMode(LED_PIN, OUTPUT);
  
}

void loop()
{ 
  //read the brightness knob to set brigtness level for LED
  lightVal = map(analogRead(LIGHT_POT_PIN), 0, 1023, 0, 255);
  
  //receive from the mouse detector arduino if mouse is detected
  mouse = digitalRead(MOUSE_PIN);
  
  //receive from the mouse detector arduino if it is daytime
  dayTime = digitalRead(DAYTIME_PIN);
  
  //read if reset button is pressed
  button = digitalRead(BUTTON_PIN);
  
  //If mouse is detected, turn on light
  //and turn the two servos to open the doors
  if(mouse) {
    analogWrite(LED_PIN, lightVal);
    servo1.write(45);
    servo2.write(135);
  	Serial.println("Mouse detected!!");
    if(dayTime) playMusic(); //only play music if it is daytime
  }
  
  //If reset button pressed, turn off light and music,
  //and reset the servos back to its original position
  if(!button) {
    analogWrite(LED_PIN, 0);
    noTone(PIEZO_PIN);
    servo1.write(0);
    servo2.write(180);
    mouse = false;
    Serial.println("Reset");
  }
  
  delay(100);
}

//play a tune on the speaker 
void playMusic()
{
  tone(PIEZO_PIN, 600, 150);
  delay(150);
  tone(PIEZO_PIN, 800, 150);
  delay(150);
  tone(PIEZO_PIN, 1000, 150);
  delay(150);
  tone(PIEZO_PIN, 1200, 150);
  delay(350);
}

Schematic

Design Files

mouse_CAD

Team Beth Prototype Documentation

What we built

Our moss spreading device is a simple design. It is a remote-controlled car with a blender on the top, a tank to hold moss, and a funnel to dispense the mixture at the bottom. The purpose of the car is to cover land with a beautiful, even layer of moss with minimal effort.

Car Model

isometric view

side view

top view

front view

Controller Model

Controller Exploded View

Controller top view

TinkerCad

Each button sends a signal to the IR receiver, represented by the remote in Tinkercad.

(Subtitles may be covered by the bottom bar)

Movement demo:

Blend/dispense demo:

After some time, Beth sees that her yard is looking a little dull so she feels like beautifying it. Some of it is covered in moss and some is not. She does some research and finds that she can make moss by blending buttermilk and moss. The only problem is that her yard is extremely huge and complex with random patches of open area mixed with sporadic trees. It’s not convenient to only have one blender-full of moss at a time to cover 3 acres of land.

She grabs her remote controlled robot, places it in her yard and flips the power toggle to the on position. Beth begins to fill up the car’s tank by blending her moss and buttermilk with the blend button on the remote. After two full blenders, Beth starts to drive the robot by pressing the directional buttons on the remote and also pressing the dispense button so she could spread the moss mix however she would like.

How we got here

Vehicle design process

Here is our original idea for moving the fluid in the blender to the tank. It consisted of have a servo move a thin disk in and out of the nozzles. The servo would move the disk in while we didn’t need any fluid to be transferred. It would move the disk out of the way to let fluid through. We very slowly (but surely) realized that this solution wouldn’t work quite well due to the placement of the motor and how thin the disk would have to be.

Brainstorming using servomotor to open blender to tank

We then came up with a slightly better solution that would allow for easier placement of the servo and less spillage.

However, a problem we encountered right after was figuring out how to dispense the fluid that was transferred into the tank onto the ground without a pump. Here is picture showing a picture of our primary tank shape, which was close to a rectangular prism.

rectangular shaped tank

We were already trying to build to minimize weight which meant adding another motor for a pump would be costly. We then decided to sacrifice some efficiency with holding fluid and make the tank angled so that gravity could push out the fluid.

drawings for gravity powered tank

final tank design

One big problem we had to address was the dispensing mechanism.  We had to take into account leaking a mixture of thick-milkshake-like consistency, the weight of the liquid, and space allocation in the robot. A few ideas we had was a way to use magnets to open and close a door, gears that would use DC motors to bring a door up and down (like a garage door), and ways to use servos to pull a plane up and down. However, none of these ideas really solved the problem of massive waste while opening and closing.

Drawbridge style idea

Our solution was to mimic the blender to tank design from earlier and spread the liquid with a funnel-shaped output.

funnel design for spread

The biggest breakthrough decision we made was to focus on moss dispensing; our original idea was to also attach a cordless leaf blower. The weight of a strong enough leaf blower added on to the weight of the battery and all the liquid in the tank would make the robot need a lot of power and super strong motors. Beth shared this concern and pointed us in the right direction.

Our very first design, original plan to have leaf blowing moss robot

moss dispenser only iteration

Controller design process

The controller was initially designed to be held with both hand, inspired by Nintendo controller.

first controller iteration

However, after the feedback from Beth that she needs to spare a hand for other tasks, we changed the design to one-hand controller and made sure that the form can fit nicely in one hand.

second controller iteration

Work ethic and pace

Here is a schedule we had set for each other to get all the components of the project done.

set work schedule

We all stayed on track and it was because we spread out the tasks evenly and keep in mind what would take longer. It also helps that we made our tasks modular. Once one task was done it served as a bolster to completing the following task.

Conclusions and lessons learned

A question was brought up about the cleaning of this device: “If there is buttermilk left inside then there will be mold forming inside after a while. Will that affect future usage of the machine?” To clean the device, Beth would fill up the blender with water, blend, and let the water run through the tank. She could run water through until the dispensed water is clean.

Another comment was: “Have you thought about making the car solar power?” This is a great idea. To not add to the weight of the robot itself, we can make a solar powered charging station. But since Beth said she doesn’t get much sun, we would still have the AC adapter option.

A lot of comments were about testing:

“When the blender is full is the device top heavy and unstable?”

“How are you ensuring that your motors will be able to carry the weight of the entire device + when it’s full?”

“This is a clever design, however I worry about the vibration of the blender and how that will impact other components of your moss rover.”

The vibration of the blender is actually something that we didn’t think about. It is a valid concern that would require testing to address. We think it was unfortunate that we couldn’t test these problems but if we find the right parts, like a strong battery and DC motors, our design would be effective. If we build up a physical prototype, Beth said we can head out to her yard and test out the device and we can answer these questions then. Overall, working with Beth was a lot of fun. It was cool that we were able to build a device for a very unique problem.

We had several Zoom meetings to figure out the mechanism together (as shown in all those sketchy sketches of the robot model above). These meetings were very fruitful and effective in coming up with theoretical solutions for some technical difficulties, such as how to design the stoppers and funnels. However, because we were working remotely, we didn’t have the opportunity to test out the area our tank could cover and how effective different batteries, motors, blenders, and wheels would be. Also we were unable to test the stoppers between the blender and tank and the funnel which are important parts of our design. Also because we couldn’t meet, we divided the remote work quite clearly and were each responsible for a certain part. In retrospect, this was effective but overall, communication, about the form, electronics, and prototyping, would have been much easier in person.

We learned how to balance ambition with practicality during our prototyping process. Our first iteration, the leaf blowing moss dispenser, was a really clever idea to us, but Beth was concerned about the practicality. So for our second iteration, we made our scope more realistic. In retrospect, we should have made our process more modular from the start. If we broke down our ideas into smaller components in the beginning and made a Gantt-chart-like plan, we would have been able to see how realistic it actually was.

The biggest takeaway from our experience working with an elderly person was that checking in with Beth during our second iteration prototyping process might have helped improve our design. As we approached the end of the project, we got very invested in how the mechanism works for the robot since we were dealing with a fairly complex system. While such investigation is very valuable and we each learned a lot from the prototyping process, we put aside the need of Beth, who was supposed to be the main focus of our design. From our previous conversations with Beth on our first idea, she pointed out how some actual circumstances were different from our assumptions so we needed to take into those real-life factors into considerations, which was one of the biggest drive for us to shift to our second iteration, making our machine less ambitious and more simplistic and functional. Similarly, it would be nice if we talk to Beth to get some feedback for our second iteration, which might push our project to an even more practical, more human-centered design result.

Technical details

Robot electronics:

Breadboard image robot

Remote electronics:

https://www.tinkercad.com/things/4zUE3ML0yFY

Breadboard image remote

RC car schematic

Remote schematic

#include <IRremote.h>
#include <Servo.h>

/* Beth's Moss Dispenser
 *  
 *  Description: A remote controlled moss blender and dispenser to help Beth with her
 *  yard beautifying processes.
 *  
 *  https://www.tinkercad.com/things/1E1FraSAW9K
 *  
 * Pin mapping:
 * 
 * pin   | mode   | description
 * ------|--------|------------
 * 11     input     IR receiver   
 * 3      output    Left Front Motor Input 1  
 * 4      output    Left Front Motor Input 2
 * 5      output    Right Front Motor Input 1  
 * 6      output    Right Front Motor Input 2 
 * 7      output    Left Back Motor Input 1  
 * 8      output    Left Back Motor Input 2
 * 9      output    Right Back Motor Input 1  
 * 10     output    Right Back Motor Input 2 
 * A5     output    Blender to tank servomotor
 * A4     output    Dispensing door servomotor
 * A3     output    Blender Motor Input 1
 * A2     output    Blender Motor Input 2
 * 
 * 
 * Credit: https://www.instructables.com/id/Arduino-Modules-L298N-Dual-H-Bridge-Motor-Controll/
 * 
*/


const int RECV_PIN = 11;
const int LFIN1 = 3;
const int LFIN2 = 4;
const int RFIN1 = 5;
const int RFIN2 = 6;
const int LBIN1 = 7;
const int LBIN2 = 8;
const int RBIN1 = 9;
const int RBIN2 = 10;
const int BLEND2TK = A5;
const int DOOR = A4;
const int BLENDIN1 = 12;
const int BLENDIN2 = 13;


Servo blenderservo;
Servo doorservo;
unsigned int blendtimer = -20000;

unsigned int lastPress = 0;

bool dispensing = false;

bool on = true;

//IR Library stuff
IRrecv irrecv(RECV_PIN);
decode_results results;

void setup()
{
  pinMode(LFIN1, OUTPUT);
  pinMode(LFIN2, OUTPUT);
  pinMode(RFIN1, OUTPUT);
  pinMode(RFIN2, OUTPUT);
  pinMode(LBIN1, OUTPUT);
  pinMode(LBIN2, OUTPUT);
  pinMode(RBIN1, OUTPUT);
  pinMode(RBIN2, OUTPUT);
  pinMode(BLENDIN1, OUTPUT);
  pinMode(BLENDIN2, OUTPUT);

  blenderservo.attach(BLEND2TK);
  doorservo.attach(DOOR);
  
  Serial.begin(9600);

  irrecv.enableIRIn(); 
  
  blenderservo.write(0);
  doorservo.write(0);
}

void moveforward() {
  digitalWrite(LFIN1, HIGH);
  digitalWrite(LFIN2, LOW);
  digitalWrite(LBIN1, HIGH);
  digitalWrite(LBIN2, LOW);
  
  digitalWrite(RFIN1, HIGH);
  digitalWrite(RFIN2, LOW);
  digitalWrite(RBIN1, HIGH);
  digitalWrite(RBIN2, LOW);
}

void movebackward() {
  digitalWrite(LFIN1, LOW);
  digitalWrite(LFIN2, HIGH);
  digitalWrite(LBIN1, LOW);
  digitalWrite(LBIN2, HIGH);
  
  digitalWrite(RFIN1, LOW);
  digitalWrite(RFIN2, HIGH);
  digitalWrite(RBIN1, LOW);
  digitalWrite(RBIN2, HIGH);
}

void turnleft() {
  //left wheels backwards
  digitalWrite(LFIN1, LOW);
  digitalWrite(LFIN2, HIGH);
  digitalWrite(LBIN1, LOW);
  digitalWrite(LBIN2, HIGH);

  //right wheels forwards
  digitalWrite(RFIN1, HIGH);
  digitalWrite(RFIN2, LOW);
  digitalWrite(RBIN1, HIGH);
  digitalWrite(RBIN2, LOW);
}

void turnright() {
  //left wheels forwards
  digitalWrite(LFIN1, HIGH);
  digitalWrite(LFIN2, LOW);
  digitalWrite(LBIN1, HIGH);
  digitalWrite(LBIN2, LOW);

  //right wheels backwards
  digitalWrite(RFIN1, LOW);
  digitalWrite(RFIN2, HIGH);
  digitalWrite(RBIN1, LOW);
  digitalWrite(RBIN2, HIGH);
}

void blend() {
  digitalWrite(BLENDIN1, HIGH);
  digitalWrite(BLENDIN2, LOW);
  Serial.println("blending");
}


// volume and skip buttons move robot front, back, left, right
// 0 button dispenses moss
// 1 blends
// on/off button

void remoteaction() {
  unsigned int value = results.value;
  Serial.println(value);
  
  switch (value) {
    case 32895: {
      Serial.println("moving forward");
      moveforward();
      break;
    }
      
    case 36975: {
      Serial.println("moving backward");
      movebackward();
      break;
    }
      
    case 8415: {
      Serial.println("turning left");
      turnleft();
      break;
    }
      
    case 24735: {
      Serial.println("turning right");
      turnright();
      break;
    }
      
    case 12495: {
      Serial.println("dispensing moss");

      if (!dispensing) {
        doorservo.write(90);
        dispensing = true;
        break;
      }
      else {
        doorservo.write(0);
        dispensing = false;
        break;
      }
    }
    
    case 16580863: {
      Serial.println("on/off");

      if (on) on = false;
      else on = true;
      break;
    }
      
    case 2295: {
      Serial.println("start blending");
      blend();
      blendtimer = millis();

      break;
    }
    
      
  }
}

void loop() {
  // blend for 20 seconds (simulation different)
  if (millis() - blendtimer <= 20000) {
    blend();
  }
  // open blender to tank for 20 seconds then close (simulation different)
  if (millis() - blendtimer > 20000 && millis() - blendtimer < 40000) {
    blenderservo.write(90);
    digitalWrite(BLENDIN1, LOW);
    digitalWrite(BLENDIN2, LOW);
  }
  else {
    blenderservo.write(0);
    digitalWrite(BLENDIN1, LOW);
    digitalWrite(BLENDIN2, LOW);
  }

  // get remote button presses
  if (irrecv.decode(&results)) {
    Serial.println(results.value);
    
    remoteaction(); 
    if (!on) {
      Serial.println("off");
      digitalWrite(LFIN1, LOW);
      digitalWrite(LFIN2, LOW);
      digitalWrite(LBIN1, LOW);
      digitalWrite(LBIN2, LOW);
      digitalWrite(RFIN1, LOW);
      digitalWrite(RFIN2, LOW);
      digitalWrite(RBIN1, LOW);
      digitalWrite(RBIN2, LOW);
      digitalWrite(BLENDIN1, LOW);
      digitalWrite(BLENDIN2, LOW);
      blenderservo.write(0);
      doorservo.write(0);
      
    }
    
    irrecv.resume(); // Receive the next value
  }
  
}

/*
 * Project: Remote for Beth's Moss Dispenser
 * 
 * Description: Remote for moss dispenser
 * Imagine the Tinkercad Circuit was built using an Arduino Nano
 * 
 * https://www.tinkercad.com/things/4zUE3ML0yFY
 * 
 * Pin mapping:
 * 
 * pin   | mode       | description
 * ------|--------    |------------
 * 5      input pullup   move forward   
 * 2      input pullup   turn left 
 * 8      input pullup   turn right
 * 4      input pullup   move backward
 * 6      input pullup   blender
 * 7      input pullup   dispense door
 * 9      input pullup   on/off
 * 
 * 
 * Credit:
 * /https://www.arduino.cc/en/Tutorial/Debounce
 * https://beatyourbit.wordpress.com/2018/02/05/arduinoinfrared/
 * https://www.instructables.com/id/IR-Transmitter-and-Receiver-Using-Arduino/
 */

#include <IRremote.h>

IRsend irsend;

const int FRONT = 5;
const int LEFT = 2;
const int RIGHT = 8;
const int BACK = 4;
const int DOOR = 6;
const int BLENDER = 7;
const int ON = 9;


unsigned long lastFrontTime = 0;
unsigned long lastBackTime = 0;
unsigned long lastLeftTime = 0;
unsigned long lastRightTime = 0;
unsigned long lastBlenderTime = 0;
unsigned long lastDoorTime = 0;
unsigned long lastOnTime = 0;

unsigned long debounceDelay = 50;

int lastFrontState = HIGH; 
int lastBackState = HIGH; 
int lastLeftState = HIGH; 
int lastRightState = HIGH; 
int lastBlenderState = HIGH;
int lastDoorState = HIGH; 
int lastOnState = HIGH;

int frontState;
int backState;
int leftState;
int rightState;
int blenderState;
int doorState;
int onState;



void setup()
{
  pinMode(LEFT, INPUT_PULLUP);
  pinMode(RIGHT, INPUT_PULLUP);
  pinMode(BACK, INPUT_PULLUP);
  pinMode(FRONT, INPUT_PULLUP);
  pinMode(DOOR, INPUT_PULLUP);
  pinMode(BLENDER, INPUT_PULLUP);
  pinMode(ON, INPUT_PULLUP);
  
  Serial.begin(9600);
  
}

void loop()
{
  bool frontReading = digitalRead(FRONT);
  bool backReading = digitalRead(BACK);
  bool leftReading = digitalRead(LEFT);
  bool rightReading = digitalRead(RIGHT);
  bool doorReading = digitalRead(DOOR);
  bool blenderReading = digitalRead(BLENDER);
  bool onReading = digitalRead(ON);
  

  if (frontReading != lastFrontState) {
    lastFrontTime = millis();
  }
  if (backReading != lastBackState) {
    lastBackTime = millis();
  }
  if (leftReading != lastLeftState) {
    lastLeftTime = millis();
  }
  if (rightReading != lastRightState) {
    lastRightTime = millis();
  }
  if (blenderReading != lastBlenderState) {
    lastBlenderTime = millis();
  }
  if (doorReading != lastDoorState) {
    lastDoorTime = millis();
  }
  if (onReading != lastOnState) {
    lastOnTime = millis();
  }
  
  //checking for new press of forward button
  if ((millis() - lastFrontTime) > debounceDelay) {

    if (frontReading != frontState) {
      frontState = frontReading;
      
      if (frontState == LOW) {
        Serial.println("moving forward");
        irsend.sendNEC(0x0000807F, 32);
      }
    }
  }
  
  if ((millis() - lastBackTime) > debounceDelay) {

    if (backReading != backState) {
      backState = backReading;
      
      if (backState == LOW) {
        Serial.println("moving backward");
        irsend.sendNEC(0x0000906F, 32);
      }
    }
  }
  
  
  if ((millis() - lastLeftTime) > debounceDelay) {

    if (leftReading != leftState) {
      leftState = leftReading;
      
      if (leftState == LOW) {
        Serial.println("turning left");
        irsend.sendNEC(0x000020DF, 32);
      }
    }
  }
  
  if ((millis() - lastRightTime) > debounceDelay) {

    if (rightReading != rightState) {
      rightState = rightReading;
      
      if (rightState == LOW) {
        Serial.println("turning right");
        irsend.sendNEC(0x0000609F, 32);
      }
    }
  }
  
  
  if ((millis() - lastBlenderTime) > debounceDelay) {

    if (blenderReading != blenderState) {
      blenderState = blenderReading;
      
      if (blenderState == LOW) {
        Serial.println("blending");
        irsend.sendNEC(0x000008F7, 32);
      }
    }
  }
  
  if ((millis() - lastDoorTime) > debounceDelay) {

    if (doorReading != doorState) {
      doorState = doorReading;
      
      if (doorState == LOW) {
        Serial.println("dispense button");
        irsend.sendNEC(0x000030CF, 32);
      }
    }
  }
  
  if ((millis() - lastOnTime) > debounceDelay) {

    if (onReading != onState) {
      onState = onReading;
      
      if (onState == LOW) {
        Serial.println("turn on/off");
        irsend.sendNEC(0x00000000, 32);
      }
    }
  }
  
  lastFrontState = frontReading;
  lastBackState = backReading;
  lastLeftState = leftReading;
  lastRightState = rightReading;
  lastBlenderState = blenderReading;
  lastDoorState = doorReading;
  lastOnState = onReading;
}

For the final project of Introduction to Physical Computing, we were tasked with building an assistive device for an elderly OSHER student. Within the scope of Arduino programing and design, we were to solve a personal problem for our client through custom designs. We were introduced to our client Diane, a former teacher who works as a docent at the Pittsburgh Museum of Natural History. Through interviews with Diane and discussion among the team, we landed on the ideation and creation of a wearable metronome bracelet for Diane to help her combat her tendencies to speed up or rush.

However, due to the major world event that is the COVID-19 pandemic, the final project had to be virtualize. Instead of creating a final tangible product, we will be creating the circuitry through simulations, and CAD modelings. This limitation leads to some uncertainties in our design that can be seen throughout the documentation.

The Final Metronome Bracelet

Our team made a metronome bracelet, which vibrates based on a beat that that Diane sets. It helps Diane from restlessness and lets her  focus better on what she is doing. She enjoys doing pilates, so we intend to make this in silicone, so that it is waterproof. We worked very hard to fulfill the clean, “cool” design aspect of this device to avoid the medical-device-esque design as Diane wanted.

The overall view of the product

The front view of the final product

The dimensions of the model

See-through view for the parts

The wiring of the parts

Tinkercad interactions

important to note the presence of the breadboard is not a part of the final design. It was implemented to make clear the connections.

Once the power is turned one, the user can control the frequency of the vibration. Note that the design requires us to us a Softpot Membrane instead of a normal potentiometer. However, due to Tinkercad’s simulation limitation, this is the closest alternative. The Softpot Membrane can be seen here:

https://www.sparkfun.com/products/retired/8607

PHYSICAL MOCKUP

Note that this was made to test the behaves-like prototype in person. As such, the prototype is much rougher than the final rendering.

Overall look of the bracelet. It has 5 small extruded braille dots (simulated through buttons on fabric). These buttons can change the frequency of beats, depending on where Diane touches them.

How Diane should interact with the bracelet. Note that the braille dots are much bigger than the actual design. These were the smallest button that I had access to in the house

Slide switch for Diane to turn on and off the bracelet. Located at the end of the button range to indicate orientation of the bracelet.

“Clasp” for Diane to adjust the length of the bracelet. Note that this is a button only for physical prototyping. This would ideally be a clasp, as indicated in the final rendering.

THE NARRATIVE SKETCH

Diane is giving a crowd a tour inside the museum. As she explains the rhythms and strokes of a painting, she notices herself speeding up. But she is not sure if that is just her thought, or if she is actually talking fast. She gives a single touch under her sleeve. The bracelet quietly gives her the beats in her personalized tempo that she can pace herself to. She finishes the presentation.

Diane goes to her pilates class. The instructor tells Diane that this position requires certain pace of breathing. The instructor says “inhale on one, two, three,” yet her breathing seems to be tangled at a certain beat. Diane touches her bracelet; this time, different from the museum tour, she adjusts the beat to a slower tempo. The instructor has no idea what kind of magic this bracelet does. She does not even think that it is a medical device, because it looks so cool! Diane finishes her class successfully.

The Process

From our last prototype, our project took a big shift after talking with Diane about what she would like to see in our design. While Diane was pleased with the aesthetics of the bracelet (She emphasized that this was very important to her), she didn’t find the bracelet’s custom frequency feature to be very useful. Rather, she was more interested in having set frequencies to shift through. She preferred having around 5 modes, ranging from fast to slow. 

With this new constraint, we researched new ways of interactive inputs beyond the simple button that we had previously worked with. After some digging, Ivan was able to find a part called a softpot potentiometer that worked well with our concept. This new part is a flexible potentiometer that takes in pressure inputs across a band. We found this to be perfect for Diane to wrap around her wrist and press along the band to get different frequency outputs.

Softpot potentiometer that Ivan found to be fitting for our project. It works similarly to a real potentiometer, where you can take in a range of inputs along the band.

With this, we broke down our tasks into three parts similar to the last phase. Elizabeth took the behavioral prototype, Chloe took the looks-like, and Ivan took the works-like. Since Elizabeth and Chloe’s work were interdependent on each other, Elizabeth’s work was frontloaded in the first week to hand off to Chloe in the next week. Our main goal was to be able to put all our parts together and receive feedback from Diane before presenting on the final day, so that we can adjust any minor issues prior to the presentation. 

Our GANTT chart. Ivan – works-like. Chloe – looks-like. Elizabeth- behaves-like.

We were able to maintain the schedule and workflow quite well, despite our timeline being pushed back by a couple days. When we checked in with Diane, however, she asked for no revisions, as she was happy with how the prototype turned out, putting us back on schedule.

Behaves-like prototype

The behavioral prototype began with considering how Diane would most intuitively interact with the bracelet. One obvious, yet easy-to-miss finding was that: Even though the flex potentiometer can wrap all the way around Diane’s wrist, the top of the wrist (where the watch face would usually sit) was the only real estate that allowed interactions. The fact that the potentiometer can be the bracelet in itself was deceiving me to think that Diane can press any point in the entire bracelet. This finding was not realized until after wrapping some papers around my wrist to feel how the bracelet might feel. 

top of wrist is the only interface I can work with, contrary to prior belief that the whole bracelet can be a potentiometer

During this phase, I was flexible with exploring how to include different outputs other than the regular metronome that we had planned. One suggestion from Diane from our discussion was that she would like to see variations of beats to support her inhales and exhales during pilates. As such, I considered having two potentiometers stacked on top of each other, where the top is the regular metronome and the bottom is the irregular breathing version. After talking with my dad, he suggested a much simpler version, where you include a switch to change the two modes. 

Sketch consideration for ergonomic layouts of buttons, as well as possibility of including a “breathing” version apart from the regular metronome

While this was thought about, Ivan brought up a good point that this irregular beats feature wouldn’t be very useful in different contexts of Diane’s life. As such, we simplified our features to its most essential part, so that it is clear, easy to use and lasting over time. 

The first prototype came through a physical prototype using fabric and buttons. Main lessons from sketching the bracelet was that simplicity was crucial for a small interface. As such, I kept the bracelet quite simple with 5 buttons that would lay on top of the wrist. 

My mom testing out the bracelet. She found it very straightforward and easy to use

From testing the bracelet with my parents, and it revealed two following points:

• The band is simple and easy to understand.
• My mom wanted the metronome beat to be fast inside the wrist, and slow down away from their body, whereas my dad preferred the opposite. How do I indicate the orientation of the bracelet in an intuitive way?

With this feedback in mind, I proceeded to make the model in 3D modelling. While the last prototype dealt with clarity of interactions, I wanted the modelling to take one step further in becoming aesthetic, as Diane wants. As such, I looked into various parametric bracelets for inspiration. I found out that Rhino and Grasshopper were a particularly useful tool for this, so I decided to try them out.

Inspirations I found online of bracelets that take advantage of curve forms.

Inspirations I found online of bracelets that take advantage of curve forms.

[Source1:

https://www.grasshopper3d.com/profiles/blogs/the-ghelix-bangle-sketching-in-grasshopper-and-3d-prints?overrideMobileRedirect=1 

Source2:

https://nodeform.com/collections/bracelets/products/stainless-steel-bracelet-bangle-3d-printed-morphed-design-ready-to-ship-medium-size]

My initial attempt was on Grasshopper, to get a parametric control over the form of the bracelet. I was following this tutorial: https://www.youtube.com/watch?v=CDnTFb3BgVo 

I had some problems with the initial steps, where the “polar points” component was acting oddly. Rather than creating an arc along the desired angle, the program was creating split arcs in full rotation. I couldn’t figure out how to fix this, so I decided to drop grasshopper for the time being.

According to the tutorial, the resulting line should’ve been a line of connected dots along a curve, but my program instead created a series of disconnected lines along a circle.

After doing another round of searching, I found this tutorial to be the most helpful: https://www.youtube.com/watch?v=Q8Issn3Mpzs. It was simple enough to follow, but allowed variation of thickness along the bracelet, which was my primary goal.

Main problems came from my unfamiliarity with the software. For example, I spent a good thirty minutes trying to figure out why my line was not divided into points as described in the video. It turned out that my view settings were different from the tutorial, so I had to dig around on my own to see how to make the points appear. 

The final model. Big to small waves indicate that the frequency range goes from slow to fast. This addressed the orientation issue previously found in user testing.

While the process was quite slow as I was new to the program, I was happy with how the bracelet turned out. I liked the bulbs that formed in the waves, making it more tactile than the traditional bracelet we imagine. However, I didn’t give much attention to how electronics would be held inside, which was a critical issue that Chloe had to work around after I passed off the design to her. 

Looks-like prototype

As for rendering the final model, Chloe’s main goal was to create a simple and compact design. While Elizabeth’s 3D prototype in Rhino fit the aesthetic goals, it didn’t meet the practical goals to build off of. For one, it was difficult to maintain the wavy form all the while holding all the electronics inside. Also, the thinnest parts of the bracelet would be prone to ripping, if the bracelet were to be made of silicone as intended. As such, Chloe remodelled the bracelet to be aesthetic in a simpler form. Main constraints came from making precise housing for the exact dimensions of each electronic part. 

Chart of dimensions for each electronic part

General difficulty making the model with exact dimensions. Vibration motor is at the bottom of the wrist since the skin is the thinnest and therefore most sensitive.

To solve the issue of orientation, Chloe put a gray dot in the bracelet to indicate that this is the faster frequency where the range begins. Diane would be able to know this intuitively through extended use over time. 

Gray dot placed to indicate orientation

Works-like prototype

For building the technical part of the bracelet, Ivan’s main constraint came from the size of the bracelet, as normal electronics would be hard to fit inside. After talking with Zach, Ivan decided to use the attiny, rather than the regular Arduino uno (or even the Arduino mini), as it had the smallest form factor.

Motor doesn’t respond when the power is directly connected to the attiny. Required a transistor connected to the coin cell battery to solve the issue.

Using the attiny required some troubleshooting, however, as when it was directly connected to the vibration motor, the motor was not responding to the inputs. Suspecting that the attiny was unable to power the motor properly, Ivan tested out the attiny with an LED bulb that requires less power and found his speculation to be correct. In order to readjust from there on, he used a transistor to connect to the motor’s power from the cell battery, rather than connecting the motor’s power to the attiny.

Conclusion

The final crit gave us good insight into the strengths and weaknesses of our project. Most everyone that commented really enjoyed the simplicity and elegance of the design, and found it to be useful in a variety of contexts.

Comments from crit and our responses

“I like the sleek and compact design. However, how do you change the batteries out? Is there a door of some sort? Would it be possible to use rechargeable cells and implement wireless charging of some sort?”

Lots of concerns were raised about changing the batteries. In the final design, there would be an opening to change the batteries when necessary. I’m not sure if it would’ve been possible to make this device rechargeable, as I’m not sure what component would be used to make that. It brings up a good question of how the user knows when the thing is out of battery versus just malfunctioning. Maybe a battery indicator would’ve been a nice touch to include.

“Your device is interesting in the way that it allows you to customize frequencies. But considering that the origin of this idea was the fact that she focused on a song have you thought about including different grooves, like a swing beat, or a clave?”

This was an interesting suggestion. I think we could have gone in the direction where Diane gives us 5 theme songs that she typically has in her head and we translate them into beats… But this may require more processing power, making the bracelet less compact as we designed for. Moreover, having a simple metronome matched closer to her original intent of wanting to calm down through regular beats.

“I love the design of the first bracelet as well as the second, and Diane seems to really love this! I am still confused, is the vibration beat already programmed, and she is able to change the frequency?”

This comment was similar to one of the questions we received during the critique session, where one of the Osher students wasn’t sure how Diane would pick the range of frequency for the metronome. While we discussed with Diane that the beats shouldn’t be too fast, that was the extent of our conversation regarding the range. Perhaps there could have been another potentiometer that sets the minimum frequency…

“Nice! Diane obviously loves it–most important! I’m not a Fit Bit user, but assuming that a Fit Bit doesn’t have this functionality? Also could there be a music component, for when Diane would like to relax with other sounds and not just beats? I loved the first design with the different size dots. I also wonder if these could be good for musicians who need/use metronomes.”

From our research, FitBit doesn’t have the exact metronome functionality, but there is an existing metronome bracelet geared towards musicians. (https://www.youtube.com/watch?v=E_nLhiYdfuY) It has a very large interface and noticeable LED lights, mainly for DJs to know the beats when they are performing. Our project was meant to be surreptitious in Diane’s life, as Diane did not want something that catches someone’s attention. As for the other suggestion of having sounds, that would also go against Diane’s desire to make this device be invisible. (Even if this was somehow connected to her headphones, that would make this useless in her primary use case of working with other people.)

Overall reflection

As for the remote conditions of this project, we found ourselves having very distinct works, rather than cross collaborating as projects usually go. For instance, Elizabeth and Chloe worked on the design one after another, rather than with each other, as it was difficult to design a 3d model of something together remotely. This led to some significant design changes from one person to another, and while the design improved in the second phase, it would have been more fruitful if we had collaborated in-person. Moreover, at times when all of us wanted to discuss functionalities of the design, we ran into some frustrations where we had trouble understanding each other, as we didn’t have the usual whiteboard table to support our ideas.

As for working with Diane, we didn’t find the age gap to be too different from working with any other client, especially since Diane had a very non-age-specific problem that we had to address. (Being restless is something that is probably more common in younger people than older people, so we found it to be less about working with someone that is older, but more about someone who is full of energy.) Diane was very helpful in designing with us on the product, as she painted a very rich picture of what her life looks like, making it easy for us to make informed choices during the design phase. Diane likes having theme songs in her head. She works as a museum docent, but she can become pretty restless in public situations. She likes fashion and aesthetics, so she doesn’t want anything that screams a medical device… We connected these helpful dots early on to create the metronome, rather than some music playing device as typical “calming” products go, making our project unique but fitting for Diane’s lifestyle.

Overall, we think that we made the right choice in maintaining the simplicity of the bracelet. Throughout the process, there were some doubts within the group that the bracelet might be too simple for a final project, leading us to think of different features to add. However, all of those features were considered extraneous and taken out in the end, as a simple and clear design is much more useful than a complex and difficult one. This turned out to be a good decision, as every feedback we received mentioned how they appreciated the simplicity of the device. We hope to apply this learning in future projects – hopefully when remote conditions end, allowing us to actually see the project in real life.

Technical Details

Tinkercad

Again, it is important to note the presence of the breadboard is not a part of the final design. It was implemented to make clear the connections.

TinkerCad Breadboard Rendering

Link for Tinkercad: https://www.tinkercad.com/things/aW7kHITO1Dl

code

The code found in tinkercad can be found here:

//Wearable Metronome 
// 
//Takes an analoge input from the the pot membrane 
//and turns the input into the frequency the vibrating 
//motor would turn on. Note this code is for a ATTINY 85 
//processor 
// 
// * pin mapping: 
// * 
// * pin | mode   | description 
// * 3   | input  | potentiometer
// * 2   | output | npn transistor for vibrating motor 
//

void setup()
{
  pinMode(3, INPUT);
  pinMode(2, OUTPUT);
}

void loop()
{
  unsigned int freq = map(analogRead(3), 0, 1000, 500, 5000);
  analogWrite(2, 255);
  delay(500); // Wait for 1000 millisecond(s)
  analogWrite(2, 0);
  delay(freq); // Wait for 1000 millisecond(s)
}

However, since the tinkercad version uses a normal potentiometer, whereas our final product needs to use a softpot membrane, which doesn’t store the input, we need to change our code a little bit. We added a storage variable to keep track of the selected state of the softpot membrane.  This code hasn’t been tested due to the inability to get our hands on a softpot membrane due to the COVID-19 situation, so there could be errors with the mapping.

//Wearable Metronome
//
//Takes an analoge input from the the pot membrane
//and turns the input into the frequency the vibrating
//motor would turn on. Note this code is for a ATTINY 85
//processor
//
// * pin mapping:
// * 
// * pin  |  mode   |  description
// * 3    |  input  |  softpot membrane
// * 2    |  output |  npn transistor for vibrating motor
//

int store;
unsigned int freq;

void setup()
{
  pinMode(3, INPUT);
  pinMode(2, OUTPUT);
}

void loop()
{
  if (analogRead(3) > 5){
    store = analogRead(3);
    freq = map(store, 0, 1000, 500, 5000);
  }
  analogWrite(2, 255);
  delay(500); // Wait for 1000 millisecond(s)
  analogWrite(2, 0);
  delay(freq); // Wait for 1000 millisecond(s)
}

Circuit schematic

Circuit Diagram depicting final circuit with softpot membrane

Design files

For our final project in Introduction to Physical Computing, our group (Leah, Abel and Varsha) was given the opportunity to work with an Osher student over the course of 5 weeks. Through our interview (documented here) we met our Osher student: Jane. She is a  retired English teacher, who likes to feed birds in her backyard. While feeding the birds using her bird feeders in the backyard, Jane had a problem with pests who would often come to eat the bird food that had spilled from her feeder. She also had an issue with larger animals and birds eating too much bird food from the feeder.  So, we focused on helping her solve this problem by building a prototype (documented here) of a bird feeder that would catch the bird food that fell off the feeder and release food only to the kinds of birds she wanted. This documentation will go over the final version of this project.

What we built

Description

Our bird feeder (download zip of CAD here) uses the benefits of technology to help it solve Jane’s problems. It can be hung on a tree or post and has a 10 inch hollow cylinder to hold bird food. It uses weight sensors to determine which types of birds/animals sit on it and accordingly releases food if it is a bird that Jane wants eating from the bird feeder. Then, if bird food falls from the bird feeder it is caught by a dish handing off the bottom of the feeder. After the dish becomes heavier due to food accumulation, a LED at the top of the feeder will light up, signaling for Jane to empty out the dish.

Images of our project

Image of the assembled bird feeder

Body of the feeder

Lid of the bird feeder

Base of the feeder

Close up of the funnel dispenser mechanism

Bird food catcher

Tinkercad Interaction Piece

Physical Mockups

Below is the video of the dispenser opening and closing.

The video below shows how the lid unscrews from the feeder.

This video shows how the catcher is detachable from the base.

Below shows how the catcher moves in the wind (based off the prototype).

Mockup dish moving in the wind

Narrative Sketch

Jane comes home after dropping her daughter at work and goes to the backyard. She notices that the she doesn’t have to fill up her bird feeder as much as she did before she got this new bird feeder and sees less pests around the house. She enjoys her day outside.

Two days, later Jane does to the backyard again and notices that a light is glowing on top of the bird feeder, this means she has to empty out the dish at the bottom. She goes and grabs a cup and tilts the dish deeply to empty it out of food. She then puts that food back into the feeder and goes on with her day.

How we got here

There weren’t a lot of redesigns and edits on the technical side, but the physical design went through a lot of changes. Originally, the feeder was not designed with a funnel. Thus, seed would spill out whenever the dispenser mechanism opened. Below is what the feeder looked like before the funnel was added.

The feeder and dispenser without the funnel

To ensure the seed was slowly dispensed without making a mess I drew up the designs depicted below.

Drawing of how the funnel is supposed to work.

The feeder is designed such that the bottom of the feeder will not fill up past the funnel. The holes in the feeder are positioned such that they are just above where the seed will fill to. Thus, a bird can stick its head in and eat the seed when the dispenser opens. This ensures all the feed won’t spill out when it opens. Furthermore, the lid had to be redesigned. Since the prototype, it was planned that the lid will be attached via a hinge and latch.

The top of the feeder before I created the lid.

Upon finishing the feeder and about to start the lid, we changed the design to make it more sturdy. A screw on lid would be sturdier and less likely to detach while in use. The base of the feeder was also redesigned.

My original design of the base.

Originally, the base had a lot of detail. It looked nice, but it wasn’t necessary and it made it more difficult to attach the dish. Thus, the base was simplified in the final version.

The top of the feeder before I added the LED

The last edit made to the feeder was to add the LED. It was left to the end, so we added it on last minute. The original dish design is shown below.

We also changed the design of the bird food catcher because it would be difficult to empty out due to its thick shape.

First draft bird dish

It would also be more heavy than necessary for the bird house. The hooks will be difficult put in and take out if Jane ever wants to do that. Thus, it was redesigned to the final version seen in the final CAD model above.

Conclusions and Lessons Learned

“It’s hard to think about how this is better or worse than the many bird feeders on the market.  It really needs to be field tested.”

Since we did not have access to the lab or the relevant supplies, we were unable to test our project and determine how it compares to other feeders. We attempted to research and implement the best improvements we could think of. However, we sadly can’t say how this will face the real world.

I am concerned that the door sliding up and down might startle the birds and keep them from the feeder.

This could scare away the birds, but we believe that the motor speed can be set to a low enough value with PMW to avoid this.

I think the food mechanism is very smart and it is a very cute design. But I wonder if the bird standing area is too small, and whether the weight sensor will affect all the food slots and how do you separate them?

The weight sensor will affect all the food slots since food can be dispensed by any weight sensor.

I like the research that red is the best color. I think a common concern is making sure only birds are able to be on the platform.

Since the sensor provides an analog value, we planned on determining the threshold for a bird to prevent food from being dispensed for another animal.

• We did not face any difficulties from working remotely as group members. Each member was able to work together well online since the parts were split up. However, we did have challenges with completing this project remotely from our client. We did not get the opportunity at the beginning of the project to see her garden due to Zoom difficulties and were limited during our first interview, which was over the phone.
• One major takeaway was figuring out the questions to ask to find out problems we could build a solution for. This was a challenging aspect of the project since you have to understand the person’s routines. By understanding the client’s interests, we narrowed the scope to gardening. This experience showed us the important of understanding the routines and interests to help come up with a solution to a problem.
• Our project is physical and has to be used outside with animals. Due to the limitations of Tinkercad, we were unable to completely test the functionality of our project. We had many comments about this, so it may have been better to come up with a solution that could be easier to test. Tinkercad is better than no simulation, but it is slow and is missing many parts. If we were to do this again, we would definitely build a physical project because of the simulation limitations.

Technical Details

Tinkercad final schematic

Image of the breadboard

Circuit schematic

/*
 * Bird Feeder
 *
 * Tinkercad Circuit: https://www.tinkercad.com/things/fwrD4bi2l3y
 * 
 * Description: This code is the software we provide to power our final project bird feeder.
 * We substituted force sensitive resistors with potentiometers since Tinkercad does not have
 * force sensitive resistors. This code reads input from 4 potentiometers that represent whether
 * a bird is at one of the food openings. It also takes input from a 5th potentiometer that represents
 * whether the food plate is full. If a bird is detected and the plate is not full, food will be dispensed.
 * This code will spin the motor for one second. If the food plate is full, then a light will turn on to
 * indicate that it must be cleared before more food can be dispensed. The status of the sensors will be
 * checked every 500ms.
 * 
 * Pin mapping:
 * 
 * pin   | mode   | description
 * ------|--------|------------
 * A0     input     potentiometer/force resistor(bird) 1 
 * A1     input     potentiometer/force resistor(bird) 2 
 * A2     input     potentiometer/force resistor(bird) 3 
 * A3     input     potentiometer/force resistor(bird) 4 
 * A4     input     potentiometer/force resistor(food)
 * 3 .    output    food dispense motor
 * 12     output    LED output when food plate full
 * 
 */


const int MOTOR_PIN = 3;
const int POT1_PIN = A0;
const int POT2_PIN = A1;
const int POT3_PIN = A2;
const int POT4_PIN = A3;
const int POT5_PIN = A4;
const int LED_PIN = 12;

unsigned long timer = 0;
const int INTERVAL = 500;

const int MOTOR_SPIN = 1000;

unsigned long last_on = 0;


void setup()
{
  pinMode(MOTOR_PIN, OUTPUT);
  pinMode(POT1_PIN, INPUT);
  pinMode(POT2_PIN, INPUT);
  pinMode(POT3_PIN, INPUT);
  pinMode(POT4_PIN, INPUT);
  pinMode(POT5_PIN, INPUT);
  pinMode(LED_PIN, OUTPUT);
}

void loop()
{
  bool bird1 = analogRead(POT1_PIN) < 512;
  bool bird2 = analogRead(POT2_PIN) < 512;
  bool bird3 = analogRead(POT3_PIN) < 512;
  bool bird4 = analogRead(POT4_PIN) < 512;
  bool bird_detected = bird1 || bird2 || bird3 || bird4;
  
  bool plate_empty = analogRead(POT5_PIN) > 512;
  
  if (millis() >= timer){
    if(bird_detected && plate_empty) {
      if (millis() >= MOTOR_SPIN + last_on) {
        digitalWrite(MOTOR_PIN, HIGH);
        last_on = millis();
      } else {
        digitalWrite(MOTOR_PIN, LOW);
      }
    } else {
      digitalWrite(MOTOR_PIN, LOW);
    }
    
    if (!plate_empty) {
      digitalWrite(LED_PIN, HIGH);
    } else {
      digitalWrite(LED_PIN, LOW);
    }
    timer = millis() + INTERVAL;
  }
}

Our team’s design proposes a solution to regularizing Yale Cohen’s eating schedule. During our first discussion with Yale, we discovered that he has lived with an odd eating schedule for most of his adult life, and currently, he also has a problem with snacking.

Following our learning of Yale’s love of puzzles and etymology, we have created a device that powers on during mealtimes in Yale’s corrected eating schedule. If Yale puts the plate or bowl he ate his meal in on the device’s sensor during this time, it will reward him with a puzzle. Essentially, we are creating a device that uses a reward system to entice Yale to eat at the times he defines as acceptable.

Initial Meeting with Yale: https://courses.ideate.cmu.edu/60-223/s2020/work/initial-meeting-documentation-team-yale/

Prototype Documentation: https://courses.ideate.cmu.edu/60-223/s2020/work/team-yale-prototype-documentation/

2. What we built (5 pts.)

We built a sensor and a puzzle reward. The sensor senses whether a plate is present to check whether Yale has had his meal within the time he is supposed to and that he is adhering with good eating habits. The puzzle reward is a way to incentivize Yale to have better eating habits. When he completes his task of eating at the right time, the puzzle reward would allow Yale to choose from two different puzzles: a sequence puzzle or a hangman puzzle. For the sequence puzzle, Yale will be given three numbers and is supposed to figure out what is the last number in the sequence. For the hangman puzzle, Yale will be given the number of characters in the answer and he would have to figure out what the word is.

Physical Design

Final Design Sketch

Housing and Sensor Plate Front View

Right View

Housing and Sensor Plate Back View

Animated look at the Remote

Rendering of Device in Kitchen

Secondary Front View

Yale wakes up and heads into the kitchen. The puzzle box lights up letting him know its time to eat. He eats a bowl of cereal and places the bowl on the plate sensor. He proceeds to play a game of hangman.

Puzzle Reward

The video shows the game play of the sequence game:

The following video shows the gameplay for the hangman game:

Ultrasonic Sensor

The video shows how the circuit works: The ultrasonic sensor first senses the distance of the plates to the sensor. If the distance is less than 20cm, and the waiting time of the object is equal 10 secs, the reward for the day is displayed to Yale.

3. How we got here (5 pts.)

Puzzle Reward

Decision 1: the puzzles we wanted to include

During our initial meeting with Yale, one thing that stood out to us is how he really enjoys puzzles. We discussed the different puzzles that he liked and figured that he is both interested in words and in math. He is also very adventurous in terms of exploring new biking trails and open to any sort of puzzles.

When we were creating our initial prototypes, we had to decide which puzzle we wanted to prototype since there was a limited amount of time. Also, we want to be able to flesh out one puzzle well rather than have a bunch of half baked puzzle prototypes which would not give us good feedback from users who we are user testing on. At this point, the puzzles we were considering are:

1. Math puzzle
   • For this puzzle, we wanted to give Yale a math problem to solve since he has a strong mathematical background. However, we soon realise due to the lack of real estate on the LCD screen, it was very hard to effectively show a long math problem (such as the bridge crossing problem). Even though we are aware that there is a possibility of having the screen scroll to show more text, we think it would not be a wise decision to have too many moving digital components for someone like Yale. We then thought that having equations and sequences would be the few options that would fit into the limitation of screen width
2. Hangman
   • For this puzzle, Yale will be given a word and would have to guess what the word is. Just like hangman, he would have seven lives to guess the word. The reason we thought he might like this is mainly because he mentioned that he enjoys etymology, the study of the origin of words and the way in which their meanings have changed throughout history.
3. Mechanical puzzle
   • As we started brainstorming the different puzzles that Yale enjoys, we realise that a lot of these puzzles are just a google search away for him. This is why we wanted to think of something out of the box that is more physical. We came up with the idea of a mechanical puzzle where we will be using switches, potentiometers and tilt sensors. In order to win the game, Yale would have to place the components in the right configuration. Each time he submits his answers, instead of knowing which component is right, he will only be told how many components are correct
4. Fun facts
   • Instead of having all the rewards requiring Yale to spend time solving a problem, we wanted to provide an option where Yale is able to be rewarded without having to spend a lot of time with the system. For this option, we wanted to just display a fun fact.
5. Trivia questions
   • A middle ground between a puzzle that would take around five minutes and a fun fact thrown to Yale is questions that would take a minute or two to answer. This allows for some sort of thinking and interaction but yet would not take Yale a long time to answer. We thought of using some true or false questions or creating a series of multiple choice questions and have it be his reward.

In the end, we decided to prototype the mechanical puzzle, because it was something that we were not sure whether Yale would enjoy. After prototyping an engineering prototype and a physical prototype, we managed to talk to Yale to get his opinions. It was very vague as to whether he wanted a mechanical puzzle because he would say that it would be “interesting” because it is “something he has never seen”. This led us to a decision point whether to include the mechanical puzzle. The decision would affect the design of the device because it would require more components which might make the design more complicated.

We decided to take his vague opinions to be indicators that he was not very interested in exploring a mechanical puzzle and thus decided to work with things he was very specific about that he liked – words and math. We then pivot our rewards to include a sequence puzzle and the hangman game.

brainstorming LCD flow

Decision: components to use

After our decision to use the sequence puzzle and the hangman game as our reward, we had to look into what are the components that we want to use. The main goals of this decision are:

1. Make sure that we are able to reuse as many components across the two puzzles
2. Make sure that the components are very clear to the user
3. Make sure the flow of events is intuitive

Taking into consideration our goals, we came up with a series of options for the components we can use:

• Option 1: 1 or 4 knobs, one button → use encoder and pushbutton
  • One knob to go through numbers (0-9)
  • Three knobs to go through alphabets? (A-Z) Each knob will control around 8 alphabets
  • Button to move to the next space? And to submit
• Option 2: 2 buttons
  • One button to move to the next space and submit
  • One button to circle through options (A → Z → A)
• Option 3: 3 buttons
  • One button to move to the next space and submit
  • One button to go to the next option (A → B)
  • One button to go to the previous option (B → A)

In an ideal world where we would have access to multiple parts and are not constrained by Tinkercad components, it would be best for us to use a rotary encoder, and modify the first option that we have.

Therefore, for our components, we decided to use:

1. One knob to loop through numbers, from 0 to 9
2. One knob to loop through alphabets, from A to Z
3. One button to confirm the selection
4. One button to move to the next option, such as moving to next place in a digit or moving from the sequence puzzle option to the hangman puzzle option
5. One button to move to the previous option

components used

Sensor

Firstly, the ultrasonic sensor circuit was built with the physical Arduino components. Feed backs from the prototype helped to identify modifications like adding a small time delay (say 10 seconds) before showing the game. Another modification was to improvise the LCD display from 16×4 size that we  initially planned to use, to 16×2 available on tinkercad.

Physical coupling of the circuit

Schematic

Sketch

Iteraction and Appearance

Early design sketch with first iteration of remote

Early design sketch with Bluetooth Communication between Sensor plate and device.

The method of interaction with the design was the first problem that I had to tackle. The first ideas I had involved placing our components on the facade of the device; however, I quickly abandoned that idea because placing the buttons and knobs on the device would limit the flexibility with which Yale could interact with the device. Additionally, I was worried that being so close to the device would make it uncomfortable to use and read the LCD. My next idea was to create a remote control device. My first iteration was inspired by the design for the Nintendo GameCube controller. The reason that I used this video game controller as a point of reference is that video game controllers are generally designed ergonomically to encourage comfortable play. However, following discussions with Yale, I abandoned this iteration because he prefered to use a single-handed device. The final design came after looking at and studying some of the television remotes in my house, which were rectilinear and encouraged single-handed use.

The appearance of the device was an easier problem to solve than interaction; however, there were many iterations before the final design. The main purpose of the housing of the device initially was to host the LCD screen and the components. The purpose of the sensor plate was to sense whether or not Yale had eaten and activate the puzzles. The sensor plate accomplishes this by detecting whether or not Yale has placed a bowl or plate on it during the time in which the device is activated. As such, early on I made the decision to make it round and flat to visually complement the objects it detects. However, the housing unit changed dramatically. The first iteration was a simple box, and while it was functional and would work, it was too generic to be truly beneficial. I then tried a more organic, rounded shape to create a visual contrast between the device and the other devices in the kitchen;  however, the white on white aesthetic of the device drew the eye as it is, and the rounded shape might prove difficult to interact with. Once I abandoned the idea of placing the interactive components of the device on the housing, I was free to downsize the device and I experimented with a trapezoidal shape that allowed for the LCD screen to be read easier. This is the design for the housing I ultimately went with. At this time, I also made the decision to separate the housing from the sensor plate and connect them with a wire to encourage flexibility.

4. Conclusions and lessons learned (5 pts.)

Response to Feedback

The rendering looked great. That said, consider using a device that is internet-enabled, such as a Particle Photon, so that you can pull new puzzles down to keep the novelty from wearing off.

This is something that we really wanted to do but was unable to due to the limitations we faced when working with tinkercad. In order to combat that problem, we tried to use as much randomness as we can so that the sequence is not very predictable. Also, this would mean that we would not have to hardcode a bunch of sequences, which would take a lot of memory on the arduino. By being able to get puzzles online, we will also not be limiting the sequence puzzles to be merely based on a function. One interesting puzzle we can have is the sequence 3, 3, 5, 4, 4, _. This sequence shows the number of letters is the word one, two, three, four, five and so on, which means the answer would be 3. Being able to obtain interesting puzzles like these would be awesome, but we also have to account for the explanation for the answer of the puzzles, which would improve the usability of the puzzle.

The design is very simple and pleasing. It seems like the project will need you to program lots and lots of puzzles so it doesn’t get boring, so that might be difficult

It is true that we would need a wide variety of puzzles so that it would not get boring. I would say that the sequence puzzle might be easy to crack once Yale figures out what the equation is. Ways we have tried to make it more challenging is by limiting it to three lives and also having a time limit. Other ideas we have to make the sequence more challenging would be to have less of the sequence given, which may or may not be solvable. In terms of the hangman game, one way to make it more interesting is to incorporate different languages that still uses the alphabets.

I like how personal this intervention is to Yale’s lifestyle. It’s interesting that solving puzzles serves as a reward. I wonder whether the current input buttons are sustainable over time, as they can become cumbersome to engage with if you were to dial through every letter and number. 

It would be interesting to be able to integrate with an actual keyboard which would make typing a word so much easier. However, one thing we wanted to do is to keep the design minimal. Also, we wanted to make sure it will be a feasible project to work on using tinkercad. It would be nice to be able to possibly connect to Yale’s phone or laptop, which would make the system simple but given our expertise, this is not something feasible for us.

I love that  there are two parts connected by IR receivers and transmitters. Does that currently work in the code or is that limited by tinkercad?

Unfortunately, we were not able to have the IR receivers and transmitters working on tinkercad. We would like to get it working if we are able to work on it further because it would greatly improve the usability of the system and not have a bunch of wires running around.

The design is really nice. Are the times for the eating schedule set? I think it would be convenient if it was adjustable.

Currently, the times are not scheduled due to the restrictions we have on tinkercad. In a perfect world, we would be using the real time clock component to figure out the time of the day. It would be nice to also start customising when meal times are.

Having an endless supply of puzzles will keep it interesting and motivating.  Really like idea of connecting to the internet for more puzzle choices. The device helps with regulating meal time but does it help with the snacking problem?

Unfortunately, the snacking problem cannot be enforced but we hope that by regulating meal times, Yale would have less need to have to snack at off hours.

Working With an Older Person

Working with an older person was a unique and valuable learning experience. Something that surprised me about Yale in particular was his vast knowledge of code and some of our components due to his work in software development before he retired. This knowledge allowed him to be more involved in some of the more technical aspects of the project than I originally expected. Working with an older person with a lot of life experience was both helpful and challenging in the context of this project in particular. The Puzzle Box is an assistive device that is attempting to modify the habits of our client Yale. Designing a device that can successfully modify somebody’s habits over time is a challenging problem to tackle to begin with; however, this challenge becomes much harder when one is trying to change habits that have been set for years or even decades. However, the long life that an older person has lived means that they are very familiar and sure of their interests, their likes, and their dislikes. This came especially helpful in our design of the reward system. Yale has established his love of puzzles for many years, and while he was open to a variety of puzzles and was willing to try new things, he also knew exactly what kind of puzzles he likes and that have kept his interest for so long.  Next time we work on designing an assistive device for an older individual, we should still take advantage of their knowledge of their interests in order to design a pleasing and appropriate device.

Working Remotely

Working remotely has been unexpectedly better than we had thought. Possibly because of the amazing team and a lively client that we had. Although, we cannot for sure say it was 100 percent because it had its challenges but one major advantage we had working remotely was that it was an opportunity to think outside the box and to put into practice what we learnt in the first half of the semester. Times when we had to improvise and make some necessary adjustments. 

Some major challenges we had were scheduling meeting times to reach the client, internet connectivity issues, easy understanding of client’s needs and limited resources/parts to work with. Due to the different calendar, we had to reschedule some meeting times with the client. Also, there was a time we had difficulty understanding what the client actually wanted which we thought could have been easier if we had a real physical conversation. Also, there was a time we had internet issues while talking to the client, and that also hindered the conversation. 

Another major challenge was limitation of parts and resources. Probably because the course as a whole is project based, working remotely did not help much with that. As good as tinkercad was, it had some limitations, for example, some components we could have used if we had physical access to the lab, were not present in the tinkercad. Also, one challenge we had was a storage problem. The original idea was to have different combinations of words as puzzles for Yale, but we realized that there would be a larger memory need for that word permutations. Yale suggested a cloud service option, but that would be out of scope for the objectives of this project.  

What worked better than we expected was the cooperation among the team and designation of the work. It was unexpectedly better because despite the distance and present situation of things, each team member delivered what was expected to make things work. We could also reach out to one another for help and suggestions. 

What we might have done differently in retrospect to allow remote collaboration work more successfully would be to have a team progress report or update. With that, it would be easy to track each team member’s progress. 

5. Technical details (4 pts.)

Ultrasonic Sensor code to detect plate.

// Ultrasonic sensor code to detect plates
// This code detects the presence of the plate. Usually there is a time lag of about 10secs before the game 
// is displayed to Yale (Client). 

#include <Wire.h>
#include <LiquidCrystal_I2C.h>

/*
   Create an LCD display object called "screen" with I2C address 0x27
   which is 16 columns wide and 2 rows tall.
*/

LiquidCrystal_I2C lcd(0x27, 16, 2);

// defining the Trig and Echo pins.
const int triggerPin = 8;
const int echoPin = 9;
const int buzzer = 10;
long duration = 0;
int distance = 0;

unsigned long previous_timer = 0;
unsigned long previous_timer_bf = 0;
unsigned long interval = 10;
unsigned long breakfastTime = 20; //assuming he eats his breakfast at 20secs (o'clock) everyday
unsigned long game_display_interval = 10;
unsigned long previous_timer_reset = 0;

// function to read the distance
long readUltrasonicDistance(int triggerPin, int echoPin)
{
  pinMode(triggerPin, OUTPUT);  // Clear the trigger
  digitalWrite(triggerPin, LOW);
  delayMicroseconds(2);
  // Sets the trigger pin to HIGH state for 10 microseconds
  digitalWrite(triggerPin, HIGH);
  delayMicroseconds(10);
  digitalWrite(triggerPin, LOW);
  pinMode(echoPin, INPUT);
  // Reads the echo pin, and returns the sound wave travel time in microseconds
  return pulseIn(echoPin, HIGH);
}


void setup() {
  // initializing the lcd:
  lcd.init();
  lcd.backlight();
  Serial.begin(9600);

  // set up the LCD's number of columns and rows:
  lcd.begin(16, 2);
  pinMode(buzzer, OUTPUT);
}

void loop() {

  previous_timer = millis() / 1000;
  
  // set the cursor to column 0, line 1
  // (note: line 1 is the second row, since counting begins with 0):
  lcd.clear();
  lcd.setCursor(0, 0);
  lcd.print("Hello Yale");
  

  // measure the ping time in distance
  distance = 0.01723 * readUltrasonicDistance(2, 3);

  if (millis() / 1000 == breakfastTime) {
    lcd.clear();
    lcd.setCursor(0, 1);
    lcd.print((String)"Time 4 Breakfast");
    tone(buzzer, 1000); // Send 1KHz sound signal...
    noTone(buzzer);     // Stop sound...
    while (millis() / 1000 < previous_timer + interval) {
      //wait approx. [period] ms
    }
  }

  if (distance <= 20 && (millis() / 1000) - previous_timer_bf >= game_display_interval) {
    lcd.clear();
    lcd.setCursor(0, 1);
    lcd.print((String)"Game 4 today is:");
    while (millis() / 1000 < previous_timer + interval) {
      //wait approx. [period] ms
    }
    previous_timer_bf = millis() / 1000;
  }

}

sensor schematics

Puzzle Reward System code

// include the library code:
#include <LiquidCrystal.h>

// initialize the library with the numbers of the interface pins
LiquidCrystal lcd(12, 11, 5, 4, 3, 2);
  
const int NUM_ROT = A0;
const int ALP_ROT = A1;

const int LEFT_BUT = 6;
const int RIGHT_BUT = 7;
const int CONFIRM_BUT = 8;
const int SUBMIT_BUT = 9;

// SUBMIT BUTTON THINGS

int buttonState;
int lastButtonState = LOW;
unsigned long lastDebounceTime = 0;
unsigned long debounceDelay = 20;

// RIGHT BUTTON THINGS

int rightButtonState;
int rightLastButtonState = LOW;
unsigned long rightLastDebounceTime = 0;
unsigned long rightDebounceDelay = 20;

// HANGMAN THINGS

String alphabets[] = { "A", "B", "C", "D", "E", "F", "G", "H", "I", "J", "K", "L", "M", "N", "O", "P", "Q", "R", "S", "T", "U", "V", "W", "X", "Y", "Z"};
String words[] = {"HELLO", "THERE", "SOMETHING", "SOMETHING", "NOTHING", "LOVELY", "POOPY", "NICE", "GIRL", "GIRLFRIEND"};
const int NUM_WORDS = 10;
String answer;
String guess;
String curr;

// GENERAL THINGS

int seqLife = 3;
int hangmanLife = 7;
bool gameOver = false;
bool gameWon = false;
String gameMode;
String gameModeChoices[] = {"HANGMAN", "SEQUENCE"};
int gameModeChoice = 0;
bool gameModeChosen;

unsigned long gameTime;
const int timeLimitSecs = 300;

byte heart[8] = {
  B00000,
  B00000,
  B01010,
  B11111,
  B11111,
  B01110,
  B00100,
  B00000
};

byte clockFace[] = {
  B00000,
  B01110,
  B10101,
  B10101,
  B10111,
  B10001,
  B01110,
  B00000
};


// SEQUENCE THINGS

int seqAnswer[4] = {1, 2, 3, 4};
int seqCurr;
int seqGuess = 0;
int seqCursor = 0;

void setup() {
  lcd.createChar(0, heart);
  lcd.createChar(1, clockFace);
  lcd.begin(16, 2);
  
  pinMode(LEFT_BUT, INPUT_PULLUP);
  pinMode(RIGHT_BUT, INPUT_PULLUP);
  pinMode(CONFIRM_BUT, INPUT_PULLUP);
  pinMode(SUBMIT_BUT, INPUT_PULLUP);

  pinMode(NUM_ROT, INPUT);
  pinMode(ALP_ROT, INPUT);

  Serial.begin(9600);
  
  createHangmanAnswer();
  createSequenceAnswer();
}

void createHangmanAnswer() {
  answer = words[random(0, NUM_WORDS)];
  for (int i=0; i < answer.length(); i++) {
    guess += "_";
  }
  Serial.println(answer);
}

void createSequenceAnswer() {
  int operation1 = random(4);
  int operation2 = random(4);
  int number1 = random(1, 10);
  int number2 = random(1, 10);

  if (operation1 == 0) {
    for (int i=0; i < 4; i ++) {
      seqAnswer[i] = seqAnswer[i] + number1;
    }
  } else if (operation1 == 1) {
    for (int i=0; i < 4; i ++) {
      seqAnswer[i] = pow(seqAnswer[i], number1);
    }
  } else if (operation1 == 2) {
    for (int i=0; i < 4; i ++) {
      seqAnswer[i] = seqAnswer[i] * number1;
    }
  } else if (operation1 == 3) {
    for (int i=0; i < 4; i ++) {
      seqAnswer[i] = seqAnswer[i] - number1;
    }
  }

  if (operation2 == 0) {
    for (int i=0; i < 4; i ++) {
      seqAnswer[i] = abs(seqAnswer[i] + number2);
    }
  } else if (operation2 == 1) {
    for (int i=0; i < 4; i ++) {
      seqAnswer[i] = abs(seqAnswer[i] - number2);
    }
  } else if (operation2 == 2) {
    for (int i=0; i < 4; i ++) {
      seqAnswer[i] = abs(seqAnswer[i] * number2);
    }
  } else if (operation2 == 3) {
    for (int i=0; i < 4; i ++) {
      seqAnswer[i] = abs(pow(seqAnswer[i], number2));
    }
  }

  Serial.println((String) operation1 + " " + operation2 + " " + number1 + " " + number2);
  Serial.println((String) "sequence!: " + seqAnswer[0] + " " + seqAnswer[1] + " " + seqAnswer[2] + " " + seqAnswer[3] + " ");
}

void loop() {
//  Serial.println((String) "Your word is: " + answer);

  if (gameModeChosen) {
    int alpReading = analogRead(ALP_ROT);
    int numReading = analogRead(NUM_ROT);
  
    if (gameOver) {
      gameOverDisplay();
    } else {
      if (gameMode == "HANGMAN") {
        playHangman(alpReading);
      } else if (gameMode == "SEQUENCE") {
        playSequence(numReading);
      }
    }
  } else {
    chooseGameMode();
  }
}

/* GENERAL THINGS */

void gameOverDisplay() {
  lcd.clear();
    lcd.setCursor(0,0);
  if (!gameWon) {
    lcd.print("GAME OVER, YALE");
  } else {
    lcd.print("CONGRATS, YALE!");
  }
}

void chooseGameMode() {
  lcd.setCursor(2,0);
  lcd.print("Hangman");
  lcd.setCursor(2, 1);
  lcd.print("Sequence");

  modeRightButtonPressed();
    
  lcd.setCursor(0,gameModeChoice);
  lcd.print("-");

  modeConfirmButtonPressed();
}

void modeRightButtonPressed() {
  int reading = digitalRead(RIGHT_BUT);
  if (reading != rightLastButtonState) {
    rightLastDebounceTime = millis();
  }

  if ((millis() - rightLastDebounceTime) > rightDebounceDelay) {
    if (reading != rightButtonState) {
      rightButtonState = reading;
      if (rightButtonState == LOW) {
        lcd.clear();
        gameTime = millis();
        if (gameModeChoice == 0) {
          gameModeChoice = 1;
        } else {
          gameModeChoice = 0;
        }
      }
    }
  }
  rightLastButtonState = reading;
}

void modeConfirmButtonPressed() {
  int reading = digitalRead(CONFIRM_BUT);
  if (reading != lastButtonState) {
    lastDebounceTime = millis();
  }

  if ((millis() - lastDebounceTime) > debounceDelay) {
    if (reading != buttonState) {
      buttonState = reading;
      if (buttonState == LOW) {
        gameMode = gameModeChoices[gameModeChoice];
        gameModeChosen = true;
        lcd.clear();
      }
    }
  }
  lastButtonState = reading;
}

/* SEQUENCE GAME THINGS */

void playSequence(int numReading) {
  
  seqCurr = numReading/103;
  lcd.noBlink();
  lcd.setCursor(0, 1);
  lcd.print(seqGuess);

  lcd.setCursor(seqCursor, 1);
  lcd.print(seqCurr);
  // Serial.println((String)"current!: " + seqCurr);

  lcd.setCursor(0, 0);
  lcd.print((String) seqAnswer[0] + "," + seqAnswer[1] + "," + seqAnswer[2] + ", __");
  

  sequenceCheckRightButtonPressed();
  sequenceCheckConfirmButtonPressed();

  lcd.setCursor(12, 0);
  lcd.write(byte(0));
  lcd.setCursor(14, 0);
  lcd.print(seqLife);

  lcd.setCursor(12, 1);
  lcd.write(byte(1));
  lcd.setCursor(13, 1);
  lcd.print(timeLimitSecs-(millis() - gameTime)/1000);
}

void sequenceCheckRightButtonPressed() {
  int reading = digitalRead(RIGHT_BUT);
  if (reading != rightLastButtonState) {
    rightLastDebounceTime = millis();
  }

  if ((millis() - rightLastDebounceTime) > rightDebounceDelay) {
    if (reading != rightButtonState) {
      rightButtonState = reading;
      if (rightButtonState == LOW) {
        Serial.println((String) "NEXT NEXT " + curr);
        seqGuess = seqCurr * 10;
        seqCursor += 1;
        Serial.println("seqGuess: " + String(seqGuess));
      }
    }
  }
  rightLastButtonState = reading;
}

void sequenceCheckConfirmButtonPressed() {
  int reading = digitalRead(CONFIRM_BUT);
  if (reading != lastButtonState) {
    lastDebounceTime = millis();
  }

  if ((millis() - lastDebounceTime) > debounceDelay) {
    if (reading != buttonState) {
      buttonState = reading;
      if (buttonState == LOW) {
        Serial.println((String) "curr is: " + seqCurr);
        Serial.println((String) "guess is: " + seqGuess);
        seqGuess += seqCurr;
        Serial.println((String) "new guess is: " + seqGuess);

        sequenceSubmitGuess();
      }
    }
  }
  lastButtonState = reading;
}

void sequenceSubmitGuess() {
  if (seqAnswer[3] == seqGuess) {
    Serial.println("RIGHT ANSWER");
    gameOver = true;
    gameWon = true;
  } else {
    Serial.println("WRONG ANSWER");
    lcd.clear();
    seqLife -= 1;
    seqGuess = 0;
    seqCursor = 0;
    if (seqLife == 0) {
      gameOver = true;
    }
  }
}

/* HANGMAN THINGS */

void playHangman(int alpReading) {
  curr = alphabets[alpReading/40];
  hangmanCheckButtonPressed();
  hangmanLcdDisplay();
  delay(10); 
}

void hangmanCheckButtonPressed() {
  int reading = digitalRead(CONFIRM_BUT);
  if (reading != lastButtonState) {
    lastDebounceTime = millis();
  }

  if ((millis() - lastDebounceTime) > debounceDelay) {
    if (reading != buttonState) {
      buttonState = reading;
      if (buttonState == LOW) {
        Serial.println((String) "curr is: " + curr);
        hangmanSubmitGuess();
      }
    }
  }
  lastButtonState = reading;
}

void hangmanSubmitGuess() {
  if (answer.indexOf(curr) != -1) {
    Serial.println("you got it!");
    Serial.println((String) "curr is: " + curr);
    guess = showCharInString(guess, answer, curr);
    Serial.println((String) "new guess is: " + guess);

    if (guess.indexOf("_") == -1) {
      gameOver = true;
      gameWon = true;
    }
  } else {
    Serial.println("does not exist");
    hangmanLife -= 1;
    if (hangmanLife == 0) {
      gameOver = true;
    }
  }
}

String showCharInString(String guess, String string, String chr) {
  String result = "";
  for (int i = 0; i < string.length(); i ++) {
    if (String(string[i])  == chr || String(guess[i]) != "_") {
      result += string[i];
    } else {
      result += "_";
    }
  }
  return result;
}

void hangmanLcdDisplay() {
  if (!gameOver) {
    lcd.setCursor(0, 0);
    lcd.print(guess);
    lcd.setCursor(0, 1);
    lcd.print(curr);

    lcd.setCursor(12, 0);
    lcd.write(byte(0));
    lcd.setCursor(14, 0);
    lcd.print(hangmanLife);

    lcd.setCursor(12, 1);
    lcd.write(byte(1));
    lcd.setCursor(13, 1);
    lcd.print(timeLimitSecs-(millis() - gameTime)/1000);
  }
}

Schematics

puzzle reward schematics

What we built:

Description

Our project is a foot arrow keyboard. There is a 14 in x 10 in platform with four arrow keys arranged in one row. The keys are styled to look and function like a gas pedal in a car. On the back edge of the platform is a hinge with a kickstand, which is adjustable to allow Lynne to set the keyboard at any desired height. There are two functions to the keyboard: a computer input mode and a Simon Says mode. For the computer input mode, Lynne can connect the keyboard to her laptop and browse the internet using the arrow keys, or she can play games that only require the four arrow keys (like Tetris or Snake). For the Simon Says mode, the keyboard is preprogrammed to randomly generate arrow key patterns for Lynne to repeat after. The difficulty of the Simon Says game starts out easy with a short pattern to repeat and gets more difficult as she continues to play.

Images:

CAD Images:

Overhead view of Foot Keyboard

Right side view of Foot Keyboard

Closer view of Power and Mode switches

Close up view of Left Arrow Key

Demonstration  of joint movement for Arrow Keys and Kickstand:

TinkerCad Demo:

Narrative Sketch

After a long day at work, Lynne arrives home and considers how to spend the rest of her afternoon. She has spent the majority of the day using her hands and fingers on her work computer and the pain from her arthritis is very uncomfortable. After greeting Mendy and her two cats, she decides to give her hands a rest and occupy herself by using the Foot Keyboard! She powers on the device and plays a couple of rounds of Simon Says, to see if she can beat her record of 6 rounds from the previous day. She enjoys the fast-paced, challenging nature of the game.

A few rounds (and high scores) later, Lynne plugs the Foot Keyboard into her computer and switches the mode to ‘CPU Input’ Mode to play some online keyboard games, like Snake and Pac-man. After she feels like she has played enough games for the afternoon, she opens up her favorite online book and begins reading, using the Foot Keyboard to flip pages. When bedtime eventually rolls around, the pain in her hands has subsided, she unplugs her Foot Keyboard and prepares for bed.

How we got here:

There were multiple steps that we took to get to the final project after the initial design and prototyping such as multiple rewiring in TinkerCad, debugging the Code to work as intended, as well as changing up the design to create a believable, functional device for Lynne.

The TinkerCad wiring and the internal mechanic were made before the 3D Cad Model, which led to some mishaps as the button location didn’t match up with the model shown. Initially the ON/OFF and Mode button was a push button, but with some discussion with the team, we switched to a slide switch. In addition, before the code was added to the TinkerCad, the wiring for the push button was wired to work with a pull up input. We kept the pull up wiring and we decided to change the code to work with a pull up input. Once the design of the 3D Cad model was finished, we went back and changed the wiring to match up the location as close as possible. The Up and Down button location was swapped, as well as the indicated LED, in addition to the ON/OFF and MODE switch location. The wiring for the battery was also changed after creating the schematic, as I noticed that the ON/OFF button was not connected to the battery.

The very first draft of the TinkerCad wiring: The ON/OFF and MODE is push-button, and the arrow button is in a different order than the finalized Foot Keyboard.

For one section of the code: the computer input, we discovered that type of Arduino matters. While the code was simple with one add on of a library called “Keyboard.h” and input called “Keyboard.press” the code wasn’t able to be read with Arduino Uno. With research it was due to Uno’s different USB communication from other Arduinos. With further research Ardunio Leonardo, which has a certain USB communication that can read Keyboard and mouse inputs when connected to the computer, is the best option to use if we were to make it physically. 

Arduino Uno Port: Computer Input Code not working/verifying with multiple code errors.

Arduino Leonardo Port: Verified Code after the port has been switched to Arduino Leonardo.

Another issue we ran into was integrating the code with the Tinkercad model. To represent variables with different “states”, such as the current Simon Says game mode or the direction of a button or LED, we decided to use enums in our code. This way, we could use a switch-case statement to cleanly execute different blocks of code depending on which state a variable was. We tried to use two enums (game_stage and dir) and this compiled mess-free in the Arduino IDE, but unfortunately caused mysterious bugs when pasting the code into the Tinkercad model. Unable to figure out the issue on our own, with the help of Zach, our professor, and Harshine, our TA, we were able to get the Tinkercad model working by replacing the second enum (dir) with the use of an integer (with values 0,1,2,3).

Tinkercad would throw compilation errors when attempting to use a second enum for direction.

Switched from an enum to using an integer to represent the four directions.

Conclusions and lessons learned:

We received a lot of helpful feedback and ideas from the final critique. One interesting idea which emerged from discussion was “Have you thought about adding a handle to the device?” We thought this was an excellent suggestion, since: 1) a handle wouldn’t be too hard to add to the design (a wide rectangular hole cut from base on the opposite side of the power/mode switches) and 2) it would make moving the keyboard (perhaps from the ground to the recliner) an easier endeavour. Other helpful suggestions appeared in the written feedback. One critiquer suggested mitigating typing fatigue by using “a nicer (mechanical) keyboard that isn’t as tiring to type on (since you don’t have to bottom the keys out)?” This suggestion ties into another provided during discussion, which was “There are large pre-made keys you can purchase which would allow you to avoid using foam.” Typing fatigue is definitely an important consideration, especially for a keyboard which works the typically unused foot and calf muscles. A feature to fine-tune during actual fabrication would be the thickness of the foam beneath the key-pedals to find a balance between sensitivity and fatigue. If this balance turns out to be difficult to obtain, looking into purchasing manufactured keys would definitely be a slightly more expensive, yet reasonable, suggestion to consider. A fourth suggestion, found in the written feedback, was to add a fifth “spacebar since a lot of games also require that button.” During the design phase, we played around with the idea of adding more keys but concluded that four was an optimal number, in terms of keeping the size fairly small and avoiding typing fatigue as much as possible.

Working remotely has been fairly strange as physical projects are more easily done in person. Overall, this experience was a positive one. Coordinating with each other online wasn’t too difficult. We found most difficulty in not being able to collaborate on unshareable files together, such as the Fusion 360 model and our Arduino code. Other than those challenges, we worked fairly cohesively, and there is not much we would do differently given the chance to work together again. Our client, Lynne, was extremely fun to work with, and she easily kept in contact with us as we worked on our device. We would’ve loved to meet with her in person and show her our work face to face, but chatting with her through Zoom has still been an enjoyable experience. 

Our experience of working with an older individual was pleasant, as Lynne was very helpful in each of the steps we took, and how as a whole we were able to keep our conversation light and fun. What was difficult initially was coming up with a personalized device for Lynne from our first meeting, as many of our initial designs could have been used by the public. What helped us get through the initial challenge was, as mentioned before, how Lynne was easy to keep in contact with as we worked on our device. As we narrowed down our idea, we were able to ask additional questions, such as what games she enjoyed and what difficulties she may have with our initial foot keyboard design. Each critique we got from Lynne, we were able to slowly make the keyboard more personalized for her, which was a big help as we finalized our project. Because Lynne was tech-savvy there were no problems while working remotely, but if we were to do this again, during our initial interview, we could have asked multiple follow up questions, so we do not assume things, for example, when we initially designed it to be used on the floor rather than a recliner.

Technical details:

Design Files:

Final Project Lynne Laser Cut DXFs

TinkerCad:

TinkerCad Link: https://www.tinkercad.com/things/7FbfX3RFi6Y

TinkerCad: Foot Keyboard complete wiring

Schematic:

Foot Keyboard Final Schematic

Code:

/*
   Project Title: Foot Keyboard - Team Lynne

   Names: Sue Lee, Hojung Kim, Achilles Lin

   Description: This code controls the two modes of the Foot Keyboard. The Simon Says mode randomly
   generates sequences and asks the user to repeat the sequences in increments of 3 moves. The code
   receives user input from the 4 momentary pushbuttons and delivers output through 4 LEDs and the 
   LCD screen. The Computer Input mode allows the user to use the 4 momentary pushbuttons to send
   arrow key controls to a USB connected computer. The code continually loops to detect changes in 
   the mode switch, allowing the user to switch between modes at any point in either mode.

   Pin mapping:

   pin   | mode   | description
   ------|--------|------------
   10       input    Left Key Momentary Pushbutton
   9        input    Right Key Momentary Pushbutton
   8        input    Up Key Momentary Pushbutton
   7        input    Down Key Momentary Pushbutton
   A0       output   Left Key LED
   A1       output   Right Key LED
   A2       output   Up Key LED
   A3       output   Down Key LED
   A4       output   Mode Switch


   Credit: Randomized numbers code from https://www.arduino.cc/reference/en/language/functions/random-numbers/random/

   License Notice:
   Copyright 2020 Sue Lee, Hojung Kim, Achilles Lin

   Permission is hereby granted, free of charge, to any person obtaining
   a copy of this software and associated documentation files (the "Software"),
   to deal in the Software without restriction, including without limitation
   the rights to use, copy, modify, merge, publish, distribute, sublicense,
   and/or sell copies of the Software, and to permit persons to whom the
   Software is furnished to do so, subject to the following conditions:

  The above copyright notice and this permission notice shall be included in
  all copies or substantial portions of the Software.

  THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
  IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
  FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
  THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
  LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
  OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
  SOFTWARE.

*/

#include <LiquidCrystal.h>
#include <Keyboard.h>

LiquidCrystal lcd(12, 11, 5, 4, 3, 2);

#define LBTN_PIN 10
#define RBTN_PIN 9
#define UBTN_PIN 8
#define DBTN_PIN 7

#define LLED_PIN A0
#define RLED_PIN A1
#define ULED_PIN A2
#define DLED_PIN A3

#define MODE_PIN A4

bool prev_mode;

// Stages
enum game_stage {
  OFF,
  MODE_DISPLAY,
  PRE_START,
  ROUND_DISPLAY,
  PATTERN_DISPLAY,
  PATTERN_INPUT,
  CORRECT_FEEDBACK,
  GAMEOVER_FEEDBACK
};

/* tried to use enum but would crash tinkerCad
enum dir {
  UP,
  DOWN,
  LEFT,
  RIGHT
//  {0,1,2,3} -> {UP,DOWN,LEFT,RIGHT} respectively
}; */

// must be multiple of 3
#define MAX_PTN_LEN 30

enum game_stage curr_stage = OFF;
enum game_stage prev_stage = OFF;

int round_num = 1;
int pattern[MAX_PTN_LEN] = { }; // initialize as zeros
int pattern_index = 0;

unsigned long stage_tmr = 0;

// instructions constants
#define MODE_DISPLAY_INTVL 2000
#define ROUND_DISPLAY_INTVL 2000
#define CORRECT_DISPLAY_INTVL 3000
#define GAMEOVER_DISPLAY_INTVL 3000

// in game constants
#define PTRN_DISPLAY_INTVL 2000
#define PTRN_DISPLAY_PADDING 500
#define PTRN_INPUT_INTVL 5000

/* HELPER FUNCTIONS */

// returns the amount of time that has elapsed since stage_tmr was last set
unsigned long elapsed() {
  return millis() - stage_tmr;
}

// reset the stage_tmr with millis()
void reset_timer() {
  stage_tmr = millis();
}

// returns true if the current game stage doesn't match the previous game stage
bool new_stage() {
  return curr_stage != prev_stage;
}

// returns true if the current keyboard mode doesn't match the previous keyboard mode
bool new_mode() {
  return digitalRead(MODE_PIN) != prev_mode;
}

// clears the LCD screen and prints str onto the LCD
void clearPrintLCD(String str) {
  //insert LCD print code here
  lcd.clear();
  lcd.print(str);
}

// return the negated button state the desired key where 
// {0,1,2,3} -> {UP,DOWN,LEFT,RIGHT} respectively
bool readBtn(int key) {
  switch (key) {
    case 0:
      return !digitalRead(UBTN_PIN);
    case 1:
      return !digitalRead(DBTN_PIN);
    case 2:
      return !digitalRead(LBTN_PIN);
    case 3:
      return !digitalRead(RBTN_PIN);
  }
  return false;
}

// sets the LED output value of the desired key's LED where
// {0,1,2,3} -> {UP,DOWN,LEFT,RIGHT} respectively
void setLED(int key, bool output) {
  switch (key) {
    case 0:
      digitalWrite(ULED_PIN, output);
      break;
    case 1:
      digitalWrite(DLED_PIN, output);
      break;
    case 2:
      digitalWrite(LLED_PIN, output);
      break;
    case 3:
      digitalWrite(RLED_PIN, output);
      break;
  }
}

// sets the LED output values of all the key's LEDs
void setAllLEDs(bool output) {
  for (int i = 0; i < 4; i++) {
    setLED(i, output);
  }
}

// initializes the simon says game round number and pattern sequence
void init_game() {
  // reset round_num
  round_num = 1;

  // randomize pattern sequence
  Serial.println("Generating sequence: ");
  Serial.print("<");
  for (int i = 0; i < MAX_PTN_LEN; i++) {
    pattern[i] = random(4);
    Serial.print(pattern[i]);
    if (i < MAX_PTN_LEN - 1) {
      Serial.print(",");
    }
  }
  Serial.println(">");
}

void setup() {
  // initialize inputs/outputs
  Serial.begin(9600);

  pinMode(UBTN_PIN, INPUT_PULLUP);
  pinMode(DBTN_PIN, INPUT_PULLUP);
  pinMode(LBTN_PIN, INPUT_PULLUP);
  pinMode(RBTN_PIN, INPUT_PULLUP);

  pinMode(ULED_PIN, OUTPUT);
  pinMode(DLED_PIN, OUTPUT);
  pinMode(LLED_PIN, OUTPUT);
  pinMode(RLED_PIN, OUTPUT);

  pinMode(MODE_PIN, INPUT);

  lcd.begin(16, 2);

  // if analog input pin 0 is unconnected, random analog
  // noise will cause the call to randomSeed() to generate
  // different seed numbers each time the sketch runs.
  // randomSeed() will then shuffle the random function.
  // Note: tinkerCad generates the same "random" sequence each time
  randomSeed(analogRead(0));

  // reads the keyboard mode and initliazes global mode variables
  prev_mode = digitalRead(MODE_PIN);
}

void loop() {
  // read keyboard mode
  bool mode_switch_state = digitalRead(MODE_PIN);

  if (new_mode()) {
    if (mode_switch_state) {
      // Just entered Simon Says game mode
      // set game stage to MODE_DISPLAY
      curr_stage = MODE_DISPLAY;
    } else {
      // Just entered Computer Input mode
      prev_stage = OFF;
      clearPrintLCD("CPU INPUT MODE");
      Serial.println("Computer Input Mode");
    }
  }

  bool ubtn_state;
  bool dbtn_state;
  bool lbtn_state;
  bool rbtn_state;

  // Check if keyboard is in Simon Says Game Mode
  if (mode_switch_state) {
    switch (curr_stage) {
      // MODE_DISPLAY: print "SIMON SAYS MODE" to LCD for MODE_DISPLAY_INTVL
      case MODE_DISPLAY:
        if (new_stage()) {
          // When entering MODE_DISPLAY stage, print "SIMON SAYS MODE" to LCD
          prev_stage = curr_stage;
          reset_timer();
          clearPrintLCD("SIMON SAYS MODE");

          Serial.println("MODE DISPLAY STAGE");
        } else if (elapsed() > MODE_DISPLAY_INTVL) {
          curr_stage = PRE_START;
        }
        break;
        
      // PRE_START: print "PRESS -> TO START" until right button press (light up right LED with right button press)
      case PRE_START:
        if (new_stage()) {
          // When entering PRE_START stage, print "PRESS -> TO START" to LCD
          prev_stage = curr_stage;
          reset_timer();
          clearPrintLCD("PRESS -> TO START");

          Serial.println("PRE-START STAGE");
        }
        
        // upon right key button press: light up right LED, intialize game
        rbtn_state = readBtn(3);
        if (rbtn_state) {
          setLED(3, HIGH);
          while (rbtn_state) {
            rbtn_state = readBtn(3);
          }
          setLED(3, LOW);
          curr_stage = ROUND_DISPLAY;
          init_game();
        }
        break;
        
      // ROUND_DISPLAY: print "ROUND <round_num>" for ROUND_DISPLAY_INTVL
      case ROUND_DISPLAY:
        if (new_stage()) {
          // When entering ROUND_DISPLAY stage, print "ROUND <round_num>" to LCD
          prev_stage = curr_stage;
          reset_timer();
          clearPrintLCD("ROUND ");
          lcd.print(round_num);

          Serial.println((String)"ROUND " + round_num);
        } else if (elapsed() > ROUND_DISPLAY_INTVL) {
          curr_stage = PATTERN_DISPLAY;
        }
        break;

      // PATTERN_DISPLAY: print "SIMON SAYS..." to LCD and display <3x round_num> moves with LEDS: PTRN_DISPLAY_INTVL millis LED ON, PTRN_DISPLAY_PADDING millis LED OFF
      case PATTERN_DISPLAY:
        if (new_stage()) {
          // When entering PATTERN_DISPLAY stage: print "SIMON SAYS..." to LCD, reset pattern_index
          prev_stage = curr_stage;
          reset_timer();
          clearPrintLCD("SIMON SAYS...");
          pattern_index = 1;

          Serial.println("PATTERN_DISPLAY MODE");
        }
        
        // light up each LED corresponding to sequence
        if (pattern_index > (round_num * 3)) {
          // once all moves have been displayed for current round, move to next stage
          curr_stage = PATTERN_INPUT;
        } else if (elapsed() < PTRN_DISPLAY_INTVL) {
          // turn on LED
          setLED(pattern[pattern_index - 1], HIGH);

          Serial.println((String)"Pattern index: " + pattern_index + ", Key: " + pattern[pattern_index - 1]);
          delay(PTRN_DISPLAY_INTVL);
        } else if (elapsed() > PTRN_DISPLAY_INTVL) {
          // turn off LED
          Serial.println("Turn LED OFF");
          setLED(pattern[pattern_index - 1], LOW);
          pattern_index++;
          delay(PTRN_DISPLAY_PADDING);
          reset_timer();
        }
        break;

      // PATTERN_INPUT: print "YOUR TURN!" to LCD and accept user input of pattern, light up LEDs with button presses
      case PATTERN_INPUT:
        if (new_stage()) {
          // When entering PATTERN_DISPLAY stage: print "SIMON SAYS..." to LCD, reset pattern_index
          prev_stage = curr_stage;
          reset_timer();
          clearPrintLCD("YOUR TURN");
          pattern_index = 1;
        }
        
        if (pattern_index > (round_num * 3)) {
          // once all moves have been inputted for current round, move to next stage
          curr_stage = CORRECT_FEEDBACK;
        } else if (elapsed() > PTRN_INPUT_INTVL) {
          // game over if time runs out
          curr_stage = GAMEOVER_FEEDBACK;
        } else {
          ubtn_state = readBtn(0);
          dbtn_state = readBtn(1);
          lbtn_state = readBtn(2);
          rbtn_state = readBtn(3);

          int curr_dir = pattern[pattern_index - 1];
          int input_dir = 0;

          // check if any button has been pressed
          if (ubtn_state || dbtn_state || lbtn_state || rbtn_state) {
            if (ubtn_state) {
              Serial.println("UP");
              input_dir = 0;
            } else if (dbtn_state) {
              Serial.println("DOWN");
              input_dir = 1;
            } else if (lbtn_state) {
              Serial.println("LEFT");
              input_dir = 2;
            } else if (rbtn_state) {
              Serial.println("RIGHT");
              input_dir = 3;
            }

            // turn on corresponding key LED until button unpressed
            setLED(input_dir, HIGH);
            int pressed_btn_state = readBtn(input_dir);
            while (pressed_btn_state) {
                pressed_btn_state = readBtn(input_dir);
            }
            // turn off LED
            setLED(input_dir, LOW);

            // check if correct key was pressed
            if (curr_dir == input_dir) {
              // correct key pressed
              pattern_index++;
              reset_timer();
            } else {
              // incorrect key pressed
              curr_stage = GAMEOVER_FEEDBACK;
            }
          }
        }
        break;

      // CORRECT_FEEDBACK: print "NICE JOB!" for CORRECT_DISPLAY_INTVL
      case CORRECT_FEEDBACK:
        // print "NICE JOB!" for CORRECT_DISPLAY_INTVL
        if (new_stage()) {
          prev_stage = curr_stage;
          reset_timer();
          clearPrintLCD("NICE JOB!");

          Serial.println("CORRECT FEEDBACK STAGE");
        } else if (elapsed() > CORRECT_DISPLAY_INTVL) {
          curr_stage = ROUND_DISPLAY;
          round_num++;
        }
        break;

      // GAMEOVER_FEEDBACK: print "GAME OVER :(" and flash all LEDs on for GAMEOVER_DISPLAY_INTVL
      case GAMEOVER_FEEDBACK:        
        if (new_stage()) {
          prev_stage = curr_stage;
          reset_timer();
          clearPrintLCD("GAME OVER :(");
          setAllLEDs(HIGH);

          Serial.println("GAMEOVER FEEDBACK STAGE");
        } else if (elapsed() > GAMEOVER_DISPLAY_INTVL) {
          // turn off all LEDs
          curr_stage = PRE_START;
          setAllLEDs(LOW);
        }
        break;

      // default handles OFF stage; curr_stage should never be OFF
      default:
        clearPrintLCD("ERROR");
        Serial.println("ERROR");

        delay(500);
    }
  } else {                  // computer input mode
    Keyboard.begin(); //begin keyboard

    //Up Button
    if (readBtn(0)) {
      Keyboard.press(KEY_UP_ARROW);
      Serial.println("KEY_UP_ARROW PRESSED");
    }
    else {
      Keyboard.releaseAll();
    }

    //Down Button
    if (readBtn(1)) {
      Keyboard.press(KEY_DOWN_ARROW);
      Serial.println("KEY_DOWN_ARROW PRESSED");
    }
    else {
      Keyboard.releaseAll();
    }

    //Left Button
    if (readBtn(2)) {
      Keyboard.press(KEY_LEFT_ARROW);
      Serial.println("KEY_LEFT_ARROW PRESSED");
    }
    else {
      Keyboard.releaseAll();
    }

    //Right Button
    if (readBtn(3)) {
      Keyboard.press(KEY_RIGHT_ARROW);
      Serial.println("KEY_RIGHT_ARROW PRESSED");
    }
    else {
      Keyboard.releaseAll();
    }

    delay(200);
    Keyboard.end(); //stops keybord
  }
  
  prev_mode = mode_switch_state;
  delay(50);
}

We created an assistive device for an older adult that would improve a certain aspect of their daily routine or bring more positivity to their life. We collaborated with our Osher client, Fredrick, and communicated with him throughout the process of the project to help us figure out the design for our intervention. The problem we decided to tackle was how we could make Fredrick’s daily exercise routine more interesting by creating a guide and friend that could not only carry the routine sequence but make the process more engaging.

What We Built

Our device is a hand-sized interactive device that will act as your exercise guide and companion that will shuffle your exercise routine and encourage you to finish each session. The device will say the exercise title through speakers and it will also display the exercise title on its screen in the front. You communicate to the device when you have completed that exercise by making a loud sound (this means that the environment you are working out in should not have too many loud noises or disturbances). The device will track your overall progress with the exercise routines and at the very end, it will congratulate you that you have accomplished all the exercises.

Final Product

Our Exercise Companion device

How the device lights up

Where to change the batteries

The rotary encoder allows you to switch to a different set of routines and shuffle routine

Simulation of a button press to shuffle an exercise routine in a category

Simulation of button rotation to go to next exercise category

Animation of how the device works overall

TinkerCad

The main interaction stimulated in TinkerCAD.

The celebration when the workout is finished

Physical Mockups

A prototype of how the device could work

On one of the screens portrays a digital face

Prototype of how the device could look

Interacting with the button press

To learn more about our prototypes, click here 

Narrative Sketch

To map out all the interactions Frederick might have with the exercise companion, we developed storyboards of different use scenarios.

The exercise companion is powered on with a long press of the button. A personalized welcome message is displayed on the LCD screen and is announced on the speaker. The eyes on the device respond by blinking.

The user can move on to the next exercise by pressing the button once. The name of the exercise is announced on the speaker and LED lights of the progress bar indicates the progress through the exercise session.

Alternatively, the user can also skip to the next exercise with an audio input by saying “next” out loud.

The exercise companion celebrates with an audio and a visual message when the exercise session is complete and the lights on the progress bar are all lit up.

The user can switch between different exercise routines by turning the rotary dial on the exercise companion.

How We Got Here

We initially interviewed Fredrick and brainstormed multiple ideas with him.  In the end, we decided to pursue the exercise companion because it was the most feasible, and Fredrick also agreed that it would be something fun and useful for him.

Our Zoom interview with Fredrick

A sample of our interview

To see our full interview story click here

The basic concept was to have a device that would shuffle his exercise routines. He would press the button to shuffle the exercise routine and the device would countdown each exercise for Fredrick to follow along. There would also be LED lights to track the overall progression of the routine.  But as we got into the specifics, we changed and developed our device to better solve the problem:

A collaborative drawing done by all team members over Zoom. This incorporated all of our individual prototype designs.

1. Adding voice feedback

The first time we revealed to Fredrick what we were making, we had a button that he would be able to press every time he completed a workout. However, he raised the concern that it would be very cumbersome to get up to press the button every time, therefore, this was when we decided to use sound to go to the next workout.

Initially, we had a button but changed it to a rotary encoder to better fit Fredrick’s needs. The voice recognition would be inside the device near the speaker holes.

2. Instead of two touch screen faces on opposite ends, we used a 16×2 LCD screen

In our first final prototype, we decided to have a cubic screen with a face that acts as the “personality” of the device and changed according to the progression and accomplishment of the exercises. Another screen was in the back to have the title of the routines, in case Fredrick did not hear the exercises being called out through the speakers. However, we realized this was a little unnecessary and would add bulk to our physical size, therefore, we simplified this concept by using the form of the 16×2 LCD screen as the mouth of the device. The eyes of the mouth would be cut out from the body of the device and placed  with acrylic on top to be able to see LED lights inside. The LED lights in the eyes would then flash to animate the face.

Initial rendering of the device with two cubic screens, one in front and one in the back

Final design of device with one side of the cube with a face made from acrylic, a 16×2 LCD screen, and a LED strip progression bar. The back is empty.

The face is now constructed by essential functions of the device

3.  Adding multiple routines with a rotary encoder

We realized if we shuffled all the workout exercises, it would be difficult changing back and forth from a  standing position to a floor exercise. Therefore, this is when we incorporated workout categories for Fredrick’s exercises. Instead of shuffling all the exercises, we would shuffle the exercises in a category. Thus we replaced the button we planned to use before, to a rotary encoder, where the dial part of the rotary encoder would be able to switch between categories, and the button is able to shuffle the exercises in that category.

Using a rotary encoder instead of a button

4. Congratulatory  responses

As we developed our idea of the exercise companion, we wanted to animate the device, even more, to give it more personality. Therefore, we added features where the device would congratulate you at the end of the workout and play a song that you can choose. This detail we hope would allow the user to enjoy the device more.

To see this please look at the video of the interaction under the section, Final Product.

Conclusions and Lessons Learned

During our presentation, one of the verbal comments was that the device had “an infantile appeal” to it.  We thought this was very interesting where we did not mean to make it appear like that at all, in fact, we tried to create the device so that it would appear as a neutral face. Perhaps to make it more friendly, we could animate the expressions and have them more lifelike that way. Looking at our written feedback we also received feedback that stated our device’s appearance was “cute” and they “love the look”. This shows us that our device may have successfully taken a neutral ambiguous look and that is why people are having different reactions to it – it is up to the user to determine the mood of the face. For our case, our user is Fredrick and during our meetings with him to show our device, he enjoyed the appearance. Moving forward, it may be better to include more “faces” to make the device more lively, as pointed out in one of the comments. One of the biggest limitations, however, comes from the LCD display, which has limited ability to display graphics or animation.

In our written comments, more than one person wrote that they were afraid that the user may “game the system” by actively skipping over the exercise right away.  This may be the case, but for our client’s specific situation, he did his exercises daily anyways and only needed this tool to make his routines more interesting. The device is not meant to push him to do the exercises, therefore, this would not have been a problem. If we were to design something for the purpose of helping users exercise, then we could include a mandatory timer for each exercise.

Another comment we received was that “to display the time remaining”. We initially had this feature, but our client declined it as he told us that he wanted to take charge of how long he would spend doing an exercise. He did not want the device to rush him. We think through this project, we learned that designing a device for a one person’s needs makes the device’s functions very specific, therefore, when other’s try to imagine using it, they might not always realize the background information behind it and will not be able to relate to the device as much.

Working remotely was a new experience for all of us, and although it becomes more time-efficient without the scheduling of physical meetings and the making, communicating was more difficult. It was not always very clear what each person was doing and the pace of work was difficult to track as well. The Gantt sheet was good to help us schedule what we needed to accomplish, but what was most effective was actually meeting through the Zoom meetings. Since members of the group did not meet, accountability for accomplishing work was also very individualized, which can be difficult for people facing different situations. This experience made us realize how important physical meetings were. On the other hand, working remotely has turned out to be a better experience than we expected. Even though we cannot have in-person interaction which is important for any activity, we can still communicate effectively through technology if we have the resolution. For example, brainstorming as a group may become harder or different, but with the whiteboard feature on Zoom, it is actually quite easy and engaging.

We really enjoyed talking with the Osher students and making these interesting devices where we didn’t need to think about the cases for multiple users and just one. We hope to take our plans for this project and actually create it in real life sometime in the future and finally be able to hold our hard work in our hands.

Our biggest takeaway from this project would be seeing how important the research and communication with the users are. By talking and understanding our client, Fredrick, our design was changed and guided by his needs. Without this information, the device would not have been as useful and meaningful as it is. This would apply to our future projects as well where we want to focus more time on planning and building a strong foundation of study and research for a design project that I am doing. In addition, we have learned that everyone has their own standard of what a good device or product should be. In the process of designing, we start by envisioning the device in our perspectives. Then it is crucial to communicate with the actual users to find what is working and what is off. Oftentimes there may be features that the designers thought is important but not so by the users, or features that the users value but are not considered by the designers.

Technical Details

Tinkercad link: https://www.tinkercad.com/things/0RXfOLjEqfM

Breadboard Image

This is the annotated breadboard image

Code

/*Exercise Companion
 * Zerui Huo
 * The code makes a robot-shaped device show the exericise names through the selected workout set. It goes to the next execise when it detects sound that has volume
   over the threshold. It also shows reward when the user finishes a group of exercises or the whole set. A newpixel strip shows the progress through the workout.

 *Pin mapping:
  
     pin   | mode   | description
     ------|--------|------------
     6      INPUT     This pin represents the press button on a rotary encoder.
     7      OUTPUT    The pin that controls the Neopixel ligt strip/stick
     8      INPUT     The pin that connects to the microphone that sense any audio input from the user.
     10     OUTPUT    The pin that controls one of the two speakers.
     13     OUTPUT    The pin that controls one of the two speakers.
     A0     INPUT     The pin that represents the rotary turner on the a rotary encoder.
     A3     OUTPUT    Used as digital pin to control LEDS light.
     A4     OUTPUT    Used as digital pin to control LEDS light.
     VCC              The pin for power to go into the Arduino pro mini.Connected to a 9V battery.
     

 *Credit to example codes or reference:
  *The code that generates the custom characters in LCD is written in the help of the website https://maxpromer.github.io/LCD-Character-Creator/.
  *The code for the neopixel light strip is written by refering to the information in the website https://learn.adafruit.com/adafruit-neopixel-uberguide/arduino-library-use. It is a turtorial on how to use the neopixel library.
  

 *Note: In each workout set, exercises requiring similar pose are put into groups. Tree pose, for example, is grouped with other yoga exericise with standing pose.
        The workout sets always have the same groups of exercises,but the sequence of the groups is randomized every time. In addition, the sequence for exerices in
        each group is also randomized.
*/


//Included the needed Libraries
#include <Wire.h>
#include <LiquidCrystal.h>
#include <Adafruit_NeoPixel.h>

//Defining the "audio files" for the exerice names
#define NOTE_C3  131
#define NOTE_CS3 139
#define NOTE_D3  147
#define NOTE_DS3 156
#define NOTE_E3  165
#define NOTE_F3  175
#define NOTE_FS3 185
#define NOTE_G3  196
#define NOTE_GS3 208
#define NOTE_A3  220
#define NOTE_AS3 233
#define NOTE_B3  247
#define NOTE_C4  262
#define NOTE_CS4 277
#define NOTE_D4  294
#define NOTE_DS4 311
#define NOTE_E4  330
#define NOTE_F4  349
#define NOTE_FS4 370
#define NOTE_G4  392
#define NOTE_GS4 415
#define NOTE_A4  440
#define NOTE_AS4 466
#define NOTE_B4  494
#define NOTE_C5  523
#define NOTE_CS5 554
#define NOTE_D5  587
#define NOTE_DS5 622
#define NOTE_E5  659
#define NOTE_F5  698
#define NOTE_FS5 740
#define NOTE_G5  784
#define NOTE_GS5 831
#define NOTE_A5  880
#define NOTE_AS5 932
#define NOTE_B5  988
#define NOTE_C6  1047
#define NOTE_CS6 1109
#define NOTE_D6  1175
#define NOTE_DS6 1245
#define NOTE_E6  1319
#define NOTE_F6  1397
#define NOTE_FS6 1480
#define NOTE_G6  1568
#define NOTE_GS6 1661
#define NOTE_A6  1760
#define NOTE_AS6 1865
#define NOTE_B6  1976
#define NOTE_C7  2093
#define NOTE_CS7 2217
#define NOTE_D7  2349
#define NOTE_DS7 2489
#define NOTE_E7  2637
#define NOTE_F7  2794
#define NOTE_FS7 2960
#define NOTE_G7  3136
#define NOTE_GS7 3322
#define NOTE_A7  3520
#define NOTE_AS7 3729
#define NOTE_B7  3951
#define NOTE_C8  4186
#define NOTE_CS8 4435
#define NOTE_D8  4699
#define NOTE_DS8 4978



//Naming all the pins according to stimulation in TinkerCad
//Remember to change the pin according to your wiring
const int ROTARY_BUTTON = 6;
const int ROTARY_TURNER = A0;
const int MICROPHONE = 8;
const int SPEAKER1 = 10;
const int SPEAKER2 = 13;
const int NEOPIXELSTRIP = 7;
const int LED1 = A3;
const int LED2 = A4;

//Setting the constant ints. Fre stands for the frequency to check a specific timer. Period is the period for some events to last.
const int LED_COUNT = 6;
const int SELECTINGFRE = 100;
const int MICROPHONEFRE = 20;
const int AUDIOPERIOD = 60;
const int STRIPPERIOD = 100;

//Global varibales that change
//Timers for "run without block"
unsigned long selectingTimer = 0;
unsigned long microphoneTimer = 0;
unsigned long audioTimer = 0;
unsigned long stripTimer = 0;

int on_off = 1;
int select = 1;
int cursor = 0;
int row;
int column = 0;
int rowIncrement = 0;
int columnIncrement = 0;
int num_of_groups;

//Lists that store informations
//The set names
String sets[] = {"Yoga Set", "Weights", "Aerobics"};

//The list that tells the row number that a set starts and ends
int set_row[] = {0, 2, 4, 8};

//Example workout sets. The "0" represents a null, and the device will go to the next group of exercises in the set.
String exercise[][3] = {
  {"Jumping Jack Jumping Jack Jumping Jack", "Cobbler Pose", "Abdominal Plank"},
  {"hee", "ha", "yo"},
  {"Tree Pose", "dance", "Upward Facing Dog"},
  {"Cat-cow", "Knee-to-chest", "0"},
  {"Knees-to-chest", "Hip flexor", "Torso rotation"},
  {"Tree pose", "Full body", "Child's Pose"},
  {"Cat-cow", "Crescent warrior", "0"},
  {"Ab plank", "Cobbler Pose", "0"},
};

//Micmic the SD card that stores the audio files of the names of the exercise
int exercise_audio[][3] = {
  {NOTE_C3, NOTE_D3, NOTE_E3},
  {NOTE_F3, NOTE_G3, NOTE_A3},
  {NOTE_B3, NOTE_C4,  NOTE_D4},
  {NOTE_E4, NOTE_F4, NOTE_F5},
  {NOTE_G4, NOTE_A4,NOTE_B4},
  {NOTE_C5, NOTE_D5, NOTE_E5},
  {NOTE_F5, NOTE_G5,  NOTE_A5},
  {NOTE_B5, NOTE_C6, NOTE_D6},
};

//Creating the custom characters for the "face" of the robot
//surprise face
byte SURPRISE_06[] = {
  B00000,
  B00001,
  B00010,
  B00100,
  B01000,
  B01000,
  B01000,
  B01000
};

byte SURPRISE_07[] = {
  B01111,
  B10000,
  B00000,
  B00000,
  B00000,
  B00000,
  B00000,
  B00000
};

byte SURPRISE_08[] = {
  B11110,
  B00001,
  B00000,
  B00000,
  B00000,
  B00000,
  B00000,
  B00000
};

byte SURPRISE_09[] = {
  B00000,
  B10000,
  B01000,
  B00100,
  B00010,
  B00010,
  B00010,
  B00010
};

byte SURPRISE_16[] = {
  B01000,
  B01000,
  B01000,
  B01000,
  B00100,
  B00010,
  B00001,
  B00000
};


byte SURPRISE_17[] = {
  B00000,
  B00000,
  B00000,
  B00000,
  B00000,
  B00000,
  B10000,
  B01111
};

byte SURPRISE_18[] = {
  B00000,
  B00000,
  B00000,
  B00000,
  B00000,
  B00000,
  B00001,
  B11110
};

byte SURPRISE_19[] = {
  B00010,
  B00010,
  B00010,
  B00010,
  B00100,
  B01000,
  B10000,
  B00000
};

//smiley face
byte SMILE_04[] = {
  B00000,
  B00000,
  B00000,
  B00000,
  B00000,
  B00001,
  B00010,
  B00100
};

byte SMILE_05[] = {
  B00000,
  B00000,
  B00000,
  B01000,
  B11000,
  B00100,
  B00010,
  B00001
};


byte SMILE_010[] = {
  B00000,
  B00000,
  B00000,
  B00010,
  B00010,
  B00101,
  B01000,
  B10000
};

byte SMILE_011[] = {
  B00000,
  B00000,
  B00000,
  B00000,
  B00000,
  B00000,
  B10000,
  B01000
};

byte SMILE_16[] = {
  B10000,
  B01000,
  B00100,
  B00010,
  B00001,
  B00000,
  B00000,
  B00000
};

byte SMILE_17[] = {
  B00000,
  B00000,
  B00000,
  B00000,
  B00000,
  B10000,
  B01111,
  B00000
};

byte  SMILE_18[] = {
  B00000,
  B00000,
  B00000,
  B00000,
  B00000,
  B00001,
  B11110,
  B00000
};


byte SMILE_19[] = {
  B00001,
  B00010,
  B00100,
  B01000,
  B10000,
  B00000,
  B00000,
  B00000
};




// generating a random sequence for exerices in each exercise group
int * generate_randnumb_exercise(int min, int length) {
  static int rando[10];
  for (int i = min; i < min + length; i++) {
    rando[i - min] = i;
  }



  randomSeed(digitalRead(A5));
  for (int i = 0; i < 200; i++) {
    int rand1;
    int rand2;
    int old_rand1;
    int old_rand2;
    rand1 = random(0, length - 1);
    rand2 = random(0, length - 1);
    old_rand1 = rando[rand1];
    old_rand2 = rando[rand2];
    rando[rand1] = old_rand2;
    rando[rand2] = old_rand1;
  }
  return rando;
}

// generating a random sequence for exerice group in set1
int * generate_randnumb_set1(int min, int length) {
  static int rando2[10];
  for (int i = min; i < min + length; i++) {
    rando2[i - min] = i;
  }


  randomSeed(digitalRead(A5));
  for (int i = 0; i < 200; i++) {
    int rand1;
    int rand2;
    int old_rand1;
    int old_rand2;
    rand1 = random(0, length - 1);
    rand2 = random(0, length - 1);
    old_rand1 = rando2[rand1];
    old_rand2 = rando2[rand2];
    rando2[rand1] = old_rand2;
    rando2[rand2] = old_rand1;
  }
  return rando2;
}


// generating a random sequence for exerice group in set2
int * generate_randnumb_set2(int min, int length) {
  static int rando3[10];
  for (int i = min; i < min + length; i++) {
    rando3[i - min] = i;
  }



  randomSeed(digitalRead(A5));
  for (int i = 0; i < 200; i++) {
    int rand1;
    int rand2;
    int old_rand1;
    int old_rand2;
    rand1 = random(0, length - 1);
    rand2 = random(0, length - 1);
    old_rand1 = rando3[rand1];
    old_rand2 = rando3[rand2];
    rando3[rand1] = old_rand2;
    rando3[rand2] = old_rand1;
  }
  return rando3;
}


// generating a random sequence for exerice group in set3
int * generate_randnumb_set3(int min, int length) {
  static int rando4[10];
  for (int i = min; i < min + length; i++) {
    rando4[i - min] = i;
  }



  randomSeed(digitalRead(A5));
  for (int i = 0; i < 200; i++) {
    int rand1;
    int rand2;
    int old_rand1;
    int old_rand2;
    rand1 = random(0, length - 1);
    rand2 = random(0, length - 1);
    old_rand1 = rando4[rand1];
    old_rand2 = rando4[rand2];
    rando4[rand1] = old_rand2;
    rando4[rand2] = old_rand1;
  }
  return rando4;
}

//Pointers that refers to the sequence of exercise/exercise groups
int *sequenceExercise = generate_randnumb_exercise(0, 3);
int *sequenceSet1 = generate_randnumb_set1(set_row[0], set_row[1] - set_row[0]);
int *sequenceSet2 = generate_randnumb_set2(set_row[1], set_row[2] - set_row[1]);
int *sequenceSet3 = generate_randnumb_set3(set_row[2], set_row[3] - set_row[2]);

//set up of the lcd
const int rs = 12, en = 11, d4 = 5, d5 = 4, d6 = 3, d7 = 2;
LiquidCrystal lcd(rs, en, d4, d5, d6, d7);

//set up the neopixel strip
Adafruit_NeoPixel strip(LED_COUNT, NEOPIXELSTRIP, NEO_GRB + NEO_KHZ800);
//defining the color for the neopixel strip
uint32_t deep_sky_blue = strip.Color(0, 191, 255);


//Supplemental Functions
//Functions for selecting a workout set
//Determine which set the rotary turner is "pointing at"
int which_set () {
  int error_angle = 30;
  int potenValue = analogRead(ROTARY_TURNER);
  int set = 0;

  //Display the name of set 1 when the potentiometer is at 0 degree
  if (potenValue <= 1023 * error_angle / 300) {
    set = 1;
    return set;
  }


  else if (potenValue >= 512 - 1023 * error_angle / 300 && potenValue <= 512 + 1023 * error_angle / 300) {
    set = 2;
    return set;
  }

  //set 3
  else if (potenValue >= 1023 * (1 - 1 * error_angle / 300)) {
    set = 3;
    return set;
  }
}

//Display the name of the workout set that is currently looking at
void select_set() {
  int buttonState = digitalRead(ROTARY_BUTTON);
  int set;
  set = which_set();
  String text = sets[set - 1];
  int startingColumn = ceil((16 - text.length()) / 2);

  lcd.setCursor(2, 0);
  lcd.print("Select a Set");


  if (on_off == 1) {
    lcd.setCursor(startingColumn, 1);
    lcd.print(text);
  }
  if (on_off == 0) {
    lcd.setCursor(startingColumn, 1);
    lcd.print("               ");
  }

  //When the pressbutton on rotary encoder is pressed
  if (buttonState == LOW) {
    select = 0;

    if (set == 1) {
      row = *sequenceSet1;
      num_of_groups = set_row[1]-set_row[0];
      Serial.println(row);
    }
    if (set == 2) {
      row = *sequenceSet2;
      num_of_groups = set_row[2]-set_row[1];
      Serial.println(row);
    }
    if (set == 3) {
      row = *sequenceSet3;
      num_of_groups = set_row[3]-set_row[2];
      Serial.println(row);
    }

    lcd.clear();
    lcd.setCursor(3, 0);
    lcd.print("Lets Begin!");
    delay(1500);
    lcd.clear();
    //Call the function robot_face to show a smiley face on the lcd
    robot_face("smiley");
    delay(2000);
    lcd.clear();
    audioTimer = millis();
  }
}

//Sumplementary function for display exercise name onto the LCD display
//This function scroll the text when it is too long to display on the 16-digit screen
void scrollLeft(String string, int row) {
  int length = string.length();
  if (millis() - selectingTimer > SELECTINGFRE) {
    if (cursor + 1 <= length ) {
      cursor = cursor + 1;
    }
    if (cursor + 1 > length) {
      cursor = 0;
    }
    selectingTimer = millis();
    lcd.clear();
  }
  String text = string.substring(cursor);
  lcd.setCursor(0, row);
  lcd.print(text);
}

//Function that show the name of the current exercise
void display_text(String string) {
  int length = string.length();
  if (length <= 16) {
    lcd.setCursor(ceil((16 - length) / 2), 0);
    lcd.print(string);
  }

  if (length > 16 && length < 32) {
    int indexofSpace = string.indexOf(" ");
    String firstWord = string.substring(0, indexofSpace);
    String secondWord = string.substring(indexofSpace + 1);

    if (firstWord.length() > 16) {
      scrollLeft(firstWord, 0);
    }
    if (firstWord.length() <= 16) {
      lcd.setCursor(ceil((16 - firstWord.length()) / 2), 0);
      lcd.print(firstWord);
    }

    if (secondWord.length() > 16) {
      scrollLeft(secondWord, 1);
    }
    if (secondWord.length() <= 16) {
      lcd.setCursor(ceil((16 - secondWord.length()) / 2), 1);
      lcd.print(secondWord);
    }
  }
  if (length >= 32) {
    scrollLeft(string, 1);
  }
  
  if (string == "end") {
    lcd.setCursor(ceil((16 - 11) / 2), 0);
    lcd.print("The workout");
    lcd.setCursor(ceil((16 - 11) / 2), 1);
    lcd.print("is finished");
  }
}

//This micmics the function of the mini mp3 player, which plays a audio record of the name of the exercise.
void play_audio(int audio) {
  while (millis() - audioTimer < AUDIOPERIOD) {
  tone(SPEAKER1,audio);
  tone(SPEAKER2,audio);
  }
  noTone(SPEAKER1);
  noTone(SPEAKER2);
}

//Rolling the light on the Neopixel light strip as a reward when a exercise group is finished
void light_up_strip() {
  while (millis() - stripTimer < STRIPPERIOD) {
  int redColor = random(0, 255);
  int greenColor = random(0,255);
  int blueColor = random(0, 255);
  for (int i=0;i < 6; i++) {
  strip.setPixelColor(i,strip.Color(redColor,greenColor,blueColor));
  }
  strip.show();
  }
  strip.clear();
}

//That function that put the custom characters onto the lcd
void robot_face(String face) {
  if (face == "surprise") {
     //Create the custom characters
     lcd.createChar(0, SURPRISE_06);
     lcd.createChar(1, SURPRISE_07);
     lcd.createChar(2, SURPRISE_08);
     lcd.createChar(3, SURPRISE_09);
     lcd.createChar(4, SURPRISE_16);
     lcd.createChar(5, SURPRISE_17);
     lcd.createChar(6, SURPRISE_18);
     lcd.createChar(7, SURPRISE_19);

     //write the characters to their places
     lcd.setCursor(6, 0);
     lcd.write(byte(0));
     lcd.setCursor(7, 0);
     lcd.write(1);
     lcd.setCursor(8, 0);
     lcd.write(2);
     lcd.setCursor(9, 0);
     lcd.write(3);
      lcd.setCursor(6, 1);
     lcd.write(4);
     lcd.setCursor(7, 1);
     lcd.write(5);
     lcd.setCursor(8, 1);
     lcd.write(6);
     lcd.setCursor(9, 1);
     lcd.write(7);
  }

   if (face == "smiley") {
     //Create the custom characters
     lcd.createChar(0,  SMILE_04);
     lcd.createChar(1,  SMILE_05);
     lcd.createChar(2,  SMILE_010);
     lcd.createChar(3,  SMILE_011);
     lcd.createChar(4,  SMILE_16);
     lcd.createChar(5,  SMILE_17);
     lcd.createChar(6,  SMILE_18);
     lcd.createChar(7,  SMILE_19);
    
     //write the characters to their places
     lcd.setCursor(4, 0);
     lcd.write(byte(0));
     lcd.setCursor(5, 0);
     lcd.write(1);
     lcd.setCursor(10, 0);
     lcd.write(2);
     lcd.setCursor(11, 0);
     lcd.write(3);
     lcd.setCursor(6, 1);
     lcd.write(4);
     lcd.setCursor(7, 1);
     lcd.write(5);
     lcd.setCursor(8, 1);
     lcd.write(6);
     lcd.setCursor(9, 1);
     lcd.write(7);
  }
}

//Simple function that turn the led on and off
void ledSwith(String command) {
  if (command == "on"){
    digitalWrite(LED1,HIGH);
    digitalWrite(LED2,HIGH);
  }
  if (command == "off"){
    digitalWrite(LED1,LOW);
    digitalWrite(LED2,LOW);
  }
}

//The function that determines what happens when the workout set is finished
void end() {
  display_text("end");
  delay(2000);
  lcd.clear();
  display_text("Great Job!");
  delay(2000);
  lcd.clear();
  robot_face("smiley");
  int redColor = random(0, 255);
  int greenColor = random(0,255);
  int blueColor = random(0, 255);
  
  //The for loop rolls the lgiht on the LED light strip until the device is powered off
  for (int i=0;i < 6; i++) {
  strip.setPixelColor(i,strip.Color(redColor,greenColor,blueColor));
  strip.show(); 
  delay(400);

    if (i==5) {
      i = -1;
      redColor = random(0, 255);
      greenColor = random(0,255);
      blueColor = random(0, 255);
    }
  }
}



void show_exercise() {
  //Displaying the name of the exercise by feeding the word to function display_text()

  
  //If the selected workout set is set 1
  if (row < set_row[1]) {
    row = *(sequenceSet1 + rowIncrement);
    column = *(sequenceExercise + columnIncrement);
   

    // columnIncrement in  the if-statement should be set to the number of columns in each row
    //The if statement checks if the exercise group is finished
    if (exercise[row][column] == "0" || columnIncrement == 3) {
      rowIncrement = rowIncrement + 1;
      columnIncrement = 0;
      row = *(sequenceSet1 + rowIncrement);
      column = *(sequenceExercise + columnIncrement);

      ledSwith("on");
      robot_face("surprise");
      light_up_strip();
      lcd.clear();
      ledSwith("off");
      audioTimer = millis();
      
      //to check if the set is finished
      if (rowIncrement == set_row[1]) {
        end();
      }
    }
    display_text(exercise[row][column]);
    play_audio(exercise_audio[row][column]);
    
  }

  //If the selected workout set is set 2
  if (row >= set_row[1] && row < set_row[2]) {
    row = *(sequenceSet2 + rowIncrement);
    column = *(sequenceExercise + columnIncrement);
    

    // columnIncrement in  the if-statement should be set to the number of columns in each row
    if (exercise[row][column] == "0" || columnIncrement == 3) {
   
      rowIncrement = rowIncrement + 1;
      columnIncrement = 0;
      row = *(sequenceSet2 + rowIncrement);
      column = *(sequenceExercise + columnIncrement);
      
      ledSwith("on");
      robot_face("surprise");
      light_up_strip();
      lcd.clear();
      ledSwith("off");
      audioTimer = millis();
        
      //to check if the set is finished
      if (rowIncrement == set_row[2] - set_row[1]) {
        end();
      }
    }
    display_text(exercise[row][column]);
    play_audio(exercise_audio[row][column]);
    
  }

  //If the selected workout set is set 3
  if (row >= set_row[2] && row < set_row[3]) {
    row = *(sequenceSet3 + rowIncrement);
    column = *(sequenceExercise + columnIncrement);
   

    // columnIncrement in  the if-statement should be set to the number of columns in each row
    if (exercise[row][column] == "0" || columnIncrement == 3) {

      rowIncrement = rowIncrement + 1;
      columnIncrement = 0;
      row = *(sequenceSet3 + rowIncrement);
      column = *(sequenceExercise + columnIncrement);
      
      robot_face("surprise");
      ledSwith("on");
      light_up_strip();
      lcd.clear();
      ledSwith("off");
      audioTimer = millis();

      
      //to check if the set is finished
      if (rowIncrement == set_row[3] - set_row[2]) {
        end();
      }
    }
    display_text(exercise[row][column]);
    play_audio(exercise_audio[row][column]);
  }
}

//This funtion controls the neopixel strip to show the progress through the set
void show_progress(int num_of_groups) {
  int lightedPixel = (rowIncrement*6)/num_of_groups;
  
  for (int i=0;i < lightedPixel; i++) {
  strip.setPixelColor(i,deep_sky_blue);
  }
  strip.show();
}


void setup() {
  // Setting up the pins
  pinMode(ROTARY_BUTTON, INPUT_PULLUP);
  pinMode(ROTARY_TURNER, INPUT);
  pinMode(SPEAKER1, OUTPUT);
  pinMode(SPEAKER2, OUTPUT);
  pinMode(MICROPHONE,INPUT_PULLUP);
  pinMode(LED1, OUTPUT);
  pinMode(LED2, OUTPUT);

  //Setting up for the lcd screen
  lcd.begin(16, 2);
  lcd.setCursor(2, 0);
  lcd.print("Lets Exercise!");
  delay(1500);
  lcd.clear();

  //Initiate the Neopixel Strip
  strip.begin();
  strip.show();


  //Initiate Serial Print
  Serial.begin(9600);
}

void loop() {
  while (select == 1) {
    select_set();
    //Timer to make the chosen workout set "flash" 
    if (millis() - selectingTimer > SELECTINGFRE) {
      if (on_off == 1) {
        on_off = 0;
      }
      else {
        on_off = 1;
      }
      selectingTimer = millis();
    }
  }

  if (millis() - microphoneTimer > MICROPHONEFRE) {
    int buttonState = digitalRead(MICROPHONE);
    if (buttonState == LOW) {
      audioTimer = millis();
      stripTimer = millis();
      columnIncrement = columnIncrement + 1;
      lcd.clear();
    }
    microphoneTimer = millis();
  }
  show_exercise();
  show_progress(num_of_groups);
}

Schematic and Design Files

Schematic

Exercise Companion.stl

Our team’s design proposes a solution for regularizing Yale Cohen’s eating schedule. During our first discussion with Yale, we discovered that he has lived with an odd eating schedule for most of his adult life, and currently, he also has a problem with snacking.

Following our learning of Yale’s love of puzzles and etymology, we have created a device that powers on during mealtimes in Yale’s corrected eating schedule. If Yale puts the plate or bowl he ate his meal in on the device’s sensor during the this time, it will reward him with a puzzle or etymology related question. Essentially, we are creating a device that uses a reward system to entice Yale to eat at the times he defines as acceptable.

Our project can be divided into two main devices, the task sensor and the reward puzzle. For each of the two devices, we decided to build an ergonomic prototype and an engineering prototype.

Task Sensor

Ergonomic Prototype

This prototype was made to answer the question of how the device is interacted with.  This prototype was made with paper, pencils, and chipboard. The front-facing panel of the prototype contains the screen, which is simulated with stapled sheets of paper, small buttons, represented by small colored pieces of paper, and control knobs, represented by pencils. On the right of the device is a cardboard surface meant to represent where our client might put his finished plate or bowl after eating.

Still Images

Overall view of Device

Cardboard “Sensor Plate”

Close-up of paper “LCD Screen”

Moving Image

Gif of my mom interacting with the prototype

Process Images

Inserting pencil “potentiometers.”

Sketch of concept

Screenshot of work-shopping some display messages.

Reflection

The purpose of my creation of an ergonomic prototype of our device was to ascertain if the device would be confusing visually. I wanted to know if the overall parts of the design, the interactive interface component and the sensor plate were designed in a clear, distinct, and intuitive way;  additionally,  I wanted to do an investigation into what makes a strong component layout. After demonstrating my device to my mother, I found that my design of the interface and sensor as an integrated device was intuitive. The low flat surface of the sensor plate made it a clear surface onto which the user of the device could place their dirty plate or bowl after they eat. She also informed me that the LCD messages were clear in their meaning and sequence in response to recognizing whether or not the  user has eaten and in giving a puzzle as a reward.  However, she felt that while the layout of the buttons and knobs were visually appealing and organized, the amount of components as well as their close proximity to each other made for an un-intuitive interaction.  This point was reiterated within our Team’s discussion with Yale following our presentation on April 6.

All of the feedback I received was essential to improving the team’s design of our device. As such,  I do not plan on ignoring any of the feedback I was given. In response to the feedback, one of my primary concerns currently is improving and distilling the layout of the interactive components so that they are intuitive and easy for Yale to use.  The feedback I received in response to the prototype was expected and incredibly useful, and I hope to utilize it to move forward with our design in a positive direction.

Engineering prototype

The engineering prototype was designed to describe the placement and the functionality of the ultrasonic sensor to detect the positioning of the plate. The prototype consist of five (5) major components : arduino board, breadboard, buzzer, ultrasonic distance sensor (to sense the distance of the plate), and the 16 x 2 liquid crystal display(lcd).

Still Image

Overall circuit design

Buzzer

16 x 2 lcd

Ultrasonic distance sensor

Moving Image

The video shows how the circuit works: The sensor first senses the distance of the plates to the sensor. If the distance is less than 20cm, and the timing of the object is (in this prototype scenario) less than 10 secs, the buzzer buzzes and displays a message on the lcd to remind Yale to wash his plates.

Process Image

Design sketch

The videos below showed the critiques from two people.

/*
  LiquidCrystal Library - Hello World

 Demonstrates the use a 16x2 LCD display.  The LiquidCrystal
 library works with all LCD displays that are compatible with the
 Hitachi HD44780 driver. There are many of them out there, and you
 can usually tell them by the 16-pin interface.

 This sketch prints "Hello World!" to the LCD
 and shows the time.

  The circuit:
 * LCD RS pin to digital pin 12
 * LCD Enable pin to digital pin 11
 * LCD D4 pin to digital pin 5
 * LCD D5 pin to digital pin 4
 * LCD D6 pin to digital pin 3
 * LCD D7 pin to digital pin 2
 * LCD R/W pin to ground
 * LCD VSS pin to ground
 * LCD VCC pin to 5V
 * 10K resistor:
 * ends to +5V and ground
 * wiper to LCD VO pin (pin 3)

 Library originally added 18 Apr 2008
 by David A. Mellis
 library modified 5 Jul 2009
 by Limor Fried (http://www.ladyada.net)
 example added 9 Jul 2009
 by Tom Igoe
 modified 22 Nov 2010
 by Tom Igoe

 This example code is in the public domain.

 http://www.arduino.cc/en/Tutorial/LiquidCrystal
 */

// include the library code:
#include <LiquidCrystal.h>

// initialize the library with the numbers of the interface pins
LiquidCrystal lcd(12, 11, 5, 4, 3, 2);

const int trigPin = 6;
const int echoPin = 7;
int const buzzPin = 10;
long duration = 0;
int distance = 0;


void setup() {
  // set up the LCD's number of columns and rows:
  lcd.begin(16, 2);
  // Print a message to the LCD.
  lcd.print("hello, world!");
  
  // setting the trigpin as output and echo pin as input
  pinMode(trigPin, OUTPUT);
  pinMode(echoPin, INPUT);
   pinMode(buzzPin, OUTPUT); // buzz pin is output to control buzzering
  Serial.begin(9600);
}

void loop() {
  // set the cursor to column 0, line 1
  lcd.setCursor(0, 0); // Set the cursor on the first column and first row.
  lcd.print("Gd morning Yale!"); // Print the string "Good morning Yale!"
  lcd.setCursor(0, 1); //Set the cursor on the third column and the second row (counting starts at 0!).
  lcd.print("Time 4 breakfast");

// Clears the trigPin by setting it to LOW
  digitalWrite(trigPin, LOW);
  delayMicroseconds(2);
  // Sets the trigPin on HIGH state for 30 micro seconds
  digitalWrite(trigPin, HIGH);
  delayMicroseconds(30);
  digitalWrite(trigPin, LOW);
  // Reads the echoPin, returns the sound wave travel time in microseconds
  duration = pulseIn(echoPin, HIGH);
  // Calculating the distance
  distance = duration * 0.034 / 2;
  // Prints the distance on the Serial Monitor
  Serial.println((String)"Distance: " + distance + " cm");




  if (distance <= 10) {

    lcd.clear();
    lcd.setCursor(0, 0);
    lcd.print("Wash your plates");
    delay(5000);

    digitalWrite(buzzPin, HIGH);   // Buzz
    delay(60);

    digitalWrite(buzzPin, LOW);  // Don't buzz
    delay(60);

    lcd.clear();
    lcd.setCursor(0, 0);
    lcd.print("Reward 4 Today:");
    lcd.setCursor(0, 1);
    lcd.print("Word: Etymology");
    delay(5000);
  }
}

Reflection

From the prototype, I was able to discover that as a team, it is better we focus more on helping Yale to improve his eating habit instead of his ‘washing dishes’ habit. Which also means that the logistics of trying to detect whether he is washing his hands or washing the plates will be eliminated and hence, focusing more on the sensor positioning. One of the critique also suggested that it would be more fancy if we can incorporate ‘word pronunciation’ into the game. Also, a delay function might be needed so that it gives some timing before buzzing when a plate is detected.

Also, about the parts used, one unexpected thing I discovered was that 16 x 4 lcd is not among the parts on tinkercad, hence the game message on the lcd had to be shortened in length.

Moving forward, as a team, we agreed that instead of detecting whether the plate is washed or not, it is better we focus more on when Yale has finished eating and places his plate near the sensor before giving him a game to play rather than when he has finished washing his plates.

Puzzle Reward

Ergonomic Prototype

The prototype is part of the puzzle reward that Yale would get once he completes his task of eating at the right time.

For the puzzle reward, Yale expressed the different things he is interested in – math puzzles, logic puzzles, word puzzles. We realized that he is interested in various puzzles and it is very accessible for him by searching on the internet. This was why we wanted to prototype something that was more unique given that we are able to work with mechanical parts. We thought of making a mechanical puzzle that would allow him some variety and hope that with the prototype, we would be able to understand whether he likes mechanical puzzles and whether it would be a good reward for him.

The first part of this prototype is an ergonomic prototype to explore different ways to place the mechanical components like switches, potentiometers, buttons and LCDs.

Still Images

front view of prototype

side view of the prototype to see the knobs

side view of the prototype showing the switches

Moving Images

Video Link

Process Images

initial design sketch

brainstorming sketches

plan to prototype the switch component

Reflection

From the prototyping process of the ergonomic prototype, I realise that depending on the person using the device, there should be more distance between the potentiometer knobs. This is so that it would be less likely for the adjacent knobs to be affected when a knob is being moved. When testing out the prototype with two other people, I realised that one of them had a much easier time because her fingers were slimmer. The other person found it harder to be able to control each knob. 

Users like the front facing part of the box are very simple and contain elements that would change the LCD screen, which includes one knob and one button. Users like this simplicity and that all the switches and knobs were not in the way. Users also comment on how it is a good idea to change the color of the LED based on the configuration of the components, like whether the switch is on or off. This is so that users would not have to constantly rotate the box around when figuring the puzzle. A user commented that since more dexterity is needed for the knobs, it is a good idea to have it on the right side, which is most people’s dominant hand. This would be something we would have to talk to Yale about to confirm.

One point brought up was that the switches were slightly harder to work with and does require more effort. This is something concerning if students are already having problems with using the switches, it would be way harder for Yale to work with the switches.

Engineering Prototype

The prototype is part of the puzzle reward that Yale would get once he completes his task of eating at the right time.

The second part of this prototype is an engineering prototype to explore the potential of using a mechanical puzzle as a reward for Yale. It is also a form of exploration of the possibility of a mechanical puzzle since it is not something that is readily available in the market.

Still Image

overall layout

welcome screen

score screen

led lights as indicators

knobs and switches

Moving Image

interaction of switches

interaction of knobs

Process Images

user testing

lcd design flow sketch

#include <Adafruit_NeoPixel.h>
#include <LiquidCrystal.h>

#define PIN 13   // input pin Neopixel is attached to
#define NUMPIXELS      10 // number of neopixels in strip

Adafruit_NeoPixel pixels = Adafruit_NeoPixel(NUMPIXELS, PIN, NEO_GRB + NEO_KHZ800);

// COLOR THINGS
const uint32_t  RED = pixels.Color(255, 0, 0);
const uint32_t  YELLOW = pixels.Color(255, 255, 0);
const uint32_t  GREEN = pixels.Color(0, 255, 0);
const uint32_t  BLUE = pixels.Color(0, 0, 255);

// SWITCH THINGS
const int SWITCH_ONE = 0;
const int SWITCH_TWO = 1;
const int SWITCH_THREE = 2;
const int SWITCH_FOUR = 3;
const int SWITCH_FIVE = 4;
const int SWITCH_SIX = 5;
int switchPins[] = {SWITCH_ONE, SWITCH_TWO, SWITCH_THREE, SWITCH_FOUR, SWITCH_FIVE, SWITCH_SIX};
const int NUM_SWITCHES = 6;

// POTENTIOMETER THINGS
const int POT_ONE = A3;
const int POT_TWO = A2;
const int POT_THREE = A1;
const int POT_FOUR = A0;
int potPins[] = {POT_ONE, POT_TWO, POT_THREE, POT_FOUR};
const int NUM_POT = 4;

// BUTTON/ QUIZ THINGS
/*
const int POT_PIN = A5;
const int BUTTON_PIN = 6;
int answer[] = [HIGH, HIGH, LOW, LOW, HIGH, HIGH, 0, 1, 2, 3];
*/
// initialize the library with the numbers of the interface pins
LiquidCrystal lcd(7, 8, 9, 10, 11, 12);

void setup() {
  pixels.begin();
  
  // set up the LCD's number of columns and rows:
  lcd.begin(16, 2);
  // Print a message to the LCD.
  lcd.print("hello, world!");
  
  // SETUP SWITCHES
  for (int i = 0; i < NUM_SWITCHES; i ++) {
    pinMode(switchPins[i], INPUT_PULLUP);
  }
  
  // SETUP POTENTIOMETERS
  for (int i = 0; i < NUM_POT; i ++) {
    pinMode(potPins[i], INPUT);
  }
}

void loop() {
  lcd.setCursor(0, 1);
  lcd.print(millis() / 1000);
  
  for (int i = 0; i < NUM_SWITCHES; i ++) {
    if (digitalRead(switchPins[i]) == LOW) {
      pixels.setPixelColor(i, GREEN);
    } else {
      pixels.setPixelColor(i, RED);
    }
  }
  
  for (int i = 0; i < NUM_POT; i ++) {
    int potLEDPos = i + NUM_SWITCHES;
    int potReading = analogRead(potPins[i]);
    if (potReading > 256 * 3) {
      pixels.setPixelColor(potLEDPos, RED);
    } else if (potReading > 256 * 2) {
      pixels.setPixelColor(potLEDPos, YELLOW);
    } else if (potReading > 256) {
      pixels.setPixelColor(potLEDPos, GREEN);
    } else {
      pixels.setPixelColor(potLEDPos, BLUE);
    }
  }
  pixels.show();
}

Reflection

The prototyping process of the engineering prototype was not very smooth for a few reasons. The first was because the simulation was very far from real time and it was hard to see the process of trying out different combinations. Without the tactile elements of turning a knob and flipping a switch, it felt less satisfying and also made the process longer and harder. 

One approach the user took when solving the puzzle was to guess every single combination. This was not ideal because it would not be a puzzle but more on a tedious task that takes more time than brain power. Given this suggestion, we plan to limit the number of guesses so that Yale would have to think of smart ways to solve the puzzle. With the limitations of the tinkercad software, it is hard to have multiple LCD screens, which is why labels should be made to explicitly specify the purpose of each button. One small comment on our initial prototype was that the maximum score was not displayed so the user did not know whether the problem was solved or not. In order to address this issue, we decided to add the maximum possible score and have the button reset the puzzle when full marks are achieved.

Final Thoughts

Overall, we feel that the prototyping process was very useful in terms of getting the ball rolling and getting a sense of the feasibility and desirability of our project. It was really useful to prototype the mechanical prototype because it was something that we were not sure would be a viable puzzle project to be used a a reward for Yale. When we met Yale over the week, we managed to show him the prototype and get some initial feedback before we move on. We think that this fail fast principle would work best in the long run and prevent us from wasting time working on something that would not meet the needs and solve problems that Yale is facing. 

The biggest challenge when prototyping remotely is that communication of ideas is harder. Anishwar and Zoe were working on ergonomic prototypes and since it was hard to verbally express how they would like their prototype to look like. Anishwar was not sure how exactly the mechanical puzzle worked which was why it was slightly harder to convey the entire idea through words and images. Similarly, the task idea of detecting a plate could be implemented in different ways. It was very difficult to convey the initial idea of the brainstorming to the person prototyping the product. When working remotely with the client, it was also harder to get a sense of whether the client likes the idea. Yale was very polite and constantly praised our efforts, which was encouraging but was not very useful in terms of getting criticism on how to improve our product. It was also hard to get him to interact with the ergonomic prototype to see whether there are any difficulties moving different components.

We decided to have each team member be in charge of a single portion of the project – Ola will be working on the code for the task sensor, Zoe will be working on the code for the puzzles and Anishwar will be working on the visuals of the prototype. This gives the team members more ownership but it is harder to keep each other accountable. Also, one risk we might face is that if one portion of the project is more time consuming, the amount of work done by each member might not be equal.

One thing we hope to do differently in the future is having more communication. Even though we would have video calls outside of class time, it was still very hard to communicate ideas through video calls. I think having the habit of quickly sketching is extremely important.

We (Leah, Abel, and Varsha) were paired with a retired English teacher named Jane. On March 25th at 10:00am we interviewed Jane to get an opportunity to learn more about her and find out how we can help her with any problems she has. The results of our meeting is detailed here.  After meeting with her and brainstorming, we came up with some prototypes that we believed could help solve some of her problems. Jane is interested in feeding birds, but she worries about pests eating the feed she leaves for them. Thus, we decided to make an automated bird feeder. Below is the details of our prototype, which we split into 3 different parts that each team member worked on.

Prototype

Our group split up the entire bird feeder and did prototypes for a part of the entire birdhouse. We split the bird feeder into:

1. The housing unit for food(looks like prototype)
2. The food catcher(looks like prototype)
3.  The electronics and circuit(works like prototype)

1. The Housing Unit

The “housing unit” of the bird feeder is the feeder and base. The final product is intended to be automated. When a bird lands on the base, the feeder will automatically open and dispense seed. The prototype was made out of cardboard due to a lack of resources, but the final version of the feeder is intended to be made of acrylic or some other waterproof material. This looks like prototype was designed to help answer the design question: what should the dimensions of the bird feeder be, and how should the dispenser open to deposit seed?

Process: 

I began making the prototype by getting some cardboard out of my recycling bin.

The cardboard material I used to make the prototype

As I stated earlier, the final product will be made of acrylic or some other waterproof material. After cutting the cardboard and taping it together, I realized the size was way too large, requiring me to rip the tape off and cut it smaller.

The original size of the side of the feeder (on the left) and the strips I cut from other pieces

After fixing the problem, the housing unit prototype was finished, and I showed it to my sister for her criticism.

My sister looking over the feeder and offering her feedback.

Finished Prototype

Below are the images of the finished prototype:

Overall view of the bird feeder housing unit

The lid of the bird feeder slightly opened

The dispenser of the bird feeder while opened

The movement of the lid and dispenser while opening:

Conclusion

Jane had no criticism for my prototype, but my sister had a lot to say about it. She brought up three key areas I could improve upon for the final project. She thought the lid I made was very flimsy. She was worried that a bird would be able to easily flip it open, so she thought I should attach a latch. Furthermore, she was concerned about how I would position the feeder in the garden. She wasn’t sure if I would place it on a pole, hang it on a branch, etc. She also told me that as long as the dispenser opened it didn’t matter how it did so, but she thought making it a sliding “door” would be easier. 

Ultimately, I got the answers to my questions. The size and dimension of the feeder was good, but the dispenser and lid could use improvements. I will be incorporating all of the feedback I received because everything my sister said is completely valid. There are a lot of changes I can make to the lid and dispenser to address her criticism and make the bird feeder better. Plus, I completely forgot to include the hook on the bird feeder. These are all important elements to it, so I will be making the necessary changes according to the feedback I received.

2. The Food Catcher

The bird food catcher is a dish that hangs off of the food housing unit to catch food that falls from the feeder when birds eat from it. The prototype I made used a paper plate in place of a catching dish, and some pieces of twine. It’s purpose is to catch the food falling off of the food housing unit and

  Process:

I started out with some common materials found around the house.

The materials I had to start out with- A paper plate, some twine, and some plastic rings

For the final version of the project, the food catching dish will be deeper and heavier, but similar in shape to the paper plate. The twine will also be stronger and instead of plastic rings, there would be hooks. The intent behind having hooks, is to make it easier for Jane to remove the dish to empty out the food.

To create these “hooks”:

Cutting a slit in the plastic rings to form hook-like structure

Then insert them:

I made a slit in the paper plate and fit the “hooks” in.

Then finally, after tying the twine to these rings, I had this initial prototype:

Initial prototype- only two points of contact with “hooks” and plate

At this point, I showed my prototype to my parents, who said that it was too basic and would not last 20 minutes outside in nature. Their main critique was that it was not durable and that I would have to use some very heavy duty materials for it to be actually usable by Jane. Another major concern was how it would handle the wind. If the wind blows too much the plate would tip over and the accumulated food would fall out. This would destroy the purpose of the automated birdhouse, since Jane’s initial problem was that the spilled food was attracting pests.  My reason for showing this prototype to my parents is that they have some experience designing and building products that are used in the real world. I wanted advice regarding the design of this dish and the question of whether this would be usable by Jane to be answered. The prototype critique gave me some ideas about how to attempt to fix these issues and all of the advice given was integrated along with these ideas in fixing the prototype.

The first thing that I wanted to integrate is more support for the dish. This initial version of the prototype had 2 contact points with the hooks on the plate. When bought out in the wind, the plate would tip sharply to the side. To attempt to fix this issue, I added 2 more contact points to the dish. This seemed to make the dish more stable and move in a more circular pattern when there was wind, rather than tipping over. In order to counter faster winds, there will be a heavier dish that is not that easily tippable in place of the paper plate. In addition to that, usability of the plate for Jane needed to be improved. In the first prototype, there were only “hooks” attached at one end of the twine so that Jane could clean out the dish very easily. Then, it occurred to me that there might be a time when Jane might need to unattach the entire dish and  put it back on. So, i attached hooks to the other end of the twine as well. Overall, the prototyping critique was very useful in improving my prototype and it surprised me how much could be change in such a simple prototype.

Attached it to an existing bird feeder I already had for testing purposes

How the new prototype looks while hanging

The new prototype hanging off a tree

Final prototype, hanging off of the tree, moving in a circular motion in the wind instead of tipping over

3. The electronics

The “works like” prototype we envisioned was created in Tinkercad. It is supposed to function as an automated bird feeder. It will dispense food when a bird lands on a pressure plate. If food has already been dispensed, another pressure plate will be triggered and will prevent more food from being dispensed.

We built a prototype with Tinkercad that consists of an Arduino, two switches, motor and a breadboard. Since Tinkercad does not have pressure pad sensors, we used regular sensors to mimic the functionality. The prototype will spin the motor only if one switch is active at a time. In practice, this means that the motor would spin if only one pressure plate is active. The motor spinning will cause the bird feeder to dispense food.

After we showed Jane this prototype, she seemed to like the idea and didn’t have any critique for it. I do plan on improving the prototype by adding a delay before the switch triggers the food functionality. Currently, the motor will spin once the switch is activated. Another critique I have is what happens when the feeder runs out of food. The motor will spin if a bird is standing on the pressure plate and no food is detected on the pressure plate. So, if food runs out, it is possible for the motor to keep spinning if a bird stands on the pressure plate indefinitely. This could drain the power source of the device.

One surprise I encountered while prototyping was the lack of a pressure plate on Tinkercad, so I had to substitute the functionality with a regular flip switch.

Animation of the prototype in action

The final prototype in TInkercad

The final prototype code

Drawing of the initial prototype

Prototype to test the motor spinning

Moving Forward

The prototyping process was very constructive. We received a lot of helpful feedback for our prototypes that we can use to improve our product. We are already planning our next steps. For the works like prototype, we plan on implementing a timer so that the food doesn’t dispense immediately once the pressure plate is pressed. This will improve the functionality because currently the food is easily and quickly emptied. The looks like prototypes will be drawn out using CAD, with the necessary changes made to improve functionality. In the event that we finish the automated bird feeder with time to spare, we plan to build a system that notifies Jane when the nectar in her hummingbird feeder needs to be replaced, depending on the temperature and humidity. Jane has multiple kinds of bird feeders, and it is very important to always leave out fresh nectar in order to attract hummingbirds.

Through this process, we learned that prototyping remotely can be challenging since we are unable to physically interact with the prototype. This is a significant part of the process since you have to be able to feel and use the device in person to critique it thoroughly. Additionally, we are unable to physically put each person’s contribution together. Thus, it is difficult to conceptualize and demonstrate how the pieces will fit together when designing the product and receiving feedback on the prototypes. We are making the best of this situation with extensive communication and collaboration. Many pictures and physical demonstrations over video call make it easier to work on. We are continuing to work on our automated bird feeder, and we will be using plenty of communication to create the best product possible.