Overview

This is a prototype of a wearable device that assists in navigation by providing tactile directional instructions to the user to assist them in wayfinding while navigating unfamiliar routes in a simple outdoor environment.

Motivation

A few years ago I was driving to my new workplace for the first time and I was using a maps application on my phone to guide me in my navigation. Midway, I was confused about which turn to take and I glanced over at my phone as I was trying to figure out what turn to take while I was driving. It was then that I was pulled over by the traffic police for using my phone while driving. That’s when I thought to myself, how am I supposed to use a navigation system, that mainly relies on visual directional instructions when I’m driving if I’m supposed to keep my eyes on the road at all times?

Final Images

An overview of the tactile wearable showing the various parts.

Coin vibration motors mounted velcro adjustable rings that go around the fingers.

The tactile wearable mounted on the dominant hand of the user.

Final video

Note: Since I was unable to record the final video myself, I used the video that Harshine shot during my demo in class. (Thanks Harshine!)

Process images and review

I initially created a storyboard to draw out scenarios of use for the tactile wearable. The storyboard helped me figure out an appropriate form factor for the device as well.

Initial storyboard illustrating the use case for the device.

Throughout the process of building this device, I got to learn how to use many sensors and components that I had never used before so it was a great opportunity for me to expand my skills in physical computing.

One such component was the GPS module. Oh the GPS module… Since this was a fairly complex component to use, I had to use a pre-written code to read the signal from the GPS module. The GPS module that I was using was a Geekstory GT-U7 GPS Module GPS that had a GPS chip and an external antenna. The GPS also had a position fix LED indicator light that indicated that the GPS was searching for satellites when it was not blinking and that the GPS had fixed on a satellite by blinking every second. I downloaded the pre-written code and connected up the GPS module and the Arduino. And then there were a few problems I ran into.

The GPS was not finding a position fix

I was excited when I first connected up all the components and expected it to work right away. However, the GPS was not fixing on a satellite although I had left it connected for several minutes. I then realized that it was almost impossible to get a GPS reading sitting in the Phys Comp lab (no surprises there). So, I went outside and stood under the clear sky for several minutes, walked around even, and then the little LED started blinking!

Getting the GPS to find a position fix by taking it outside under clear skies.

Marking the GPS coordinates on a simple outdoor to create a working demo.

And then there was a delay!

After I got the GPS to find a position fix, the GPS started transmitting signals that were a bunch of latitude and longitude coordinates. However, when I would move around, the GPS wasn’t able to update the coordinates. There seemed to be a lag in the updating of the location. I tried many things to fix this, I rechecked if all the connections were good, I ensured that I was away from buildings, I restarted my Arduino, I even held up the antenna of the GPS over my head. I even checked to make sure that I had uploaded the pre-written code exactly as it was, line by line and that was the problem! I consulted with Zac about this and upon reading the code line by line, he found a pesky little delay of 1 second right at the end of the code. The delay was originally meant to set the rate at which the GPS prints the latitude and longitude coordinates however, it was also messing up the signal that the GPS was transmitting. The GPS was picking up signals from satellites at a faster rate than it was able to transmit them which caused the signals to get jumbled up. We fixed this problem by using millis instead. Lesson learned: millis for life.

The pesky delay at the end that was the root of all problems!

Reading raw data from the GPS.

The super-sensitive vibration motors

To create vibrotactile feedback in the wearable, I was using a small coin vibrational motor that had two very thin and fragile connecting leads onto which longer connecting wires had to be soldered. However, since the connection was so delicate, I was afraid of breaking the connections which made testing the prototype a huge challenge. I consulted with Zac and asked him how I could make the prototype more robust. Zac was extremely helpful and suggested I used heat shrinking material to reinforce the joints with the connections. He drew a wonderful diagram to illustrate the process which was helped me reinforce the connections and make the prototype durable.

A diagram illustrating the process of reinforcing the vibration motor with different diameters of heat shrink tubing, courtesy Zac.

The final outcome of the reinforcing the vibration motor that made the prototype suitable to test multiple times without the fear of the connections breaking.

Discussion

To present our projects, we had an in-class demo that went differently than I imagined. I had prepared to demo the device outdoors since that was the best way I could demo the device and show that it actually worked in the way I had described it would. The demo, however, was only conducted indoors which meant I had to hard code the coordinates and fake the feedback from the wearable, which I was able to successfully do. One comment I got was the “the demo could have involved a visual aspect to show what was going on in the device (to the audience)”. I agree that LED lights would have probably helped to better indicate which vibrational motors were active. Since I was prepared for a live demo, in which a volunteer would be guided from a preset origin to a destination along a simple route outdoors, I was unable to add a visual component to the demo. I intended to add LED lights to the vibrational rings for demo purposes post-spring break which I, unfortunately, couldn’t do.

Another comment I received was “It would’ve been nice to have an enclosing for the device.” I agree with this feedback, however, the challenge was that the connections hadn’t been reinforced yet so I was afraid to make anything permanent since I only had a limited number of small coin vibrational motors available. Another challenge was that the GPS unit had to have a view of the clear sky, so enclosing it in a bix might have interfered with its ability to find a position fix.

Overall, I learned so much through this project. Not only did I gain technical skills but this project also made me persevere and build the thing I intended to build no matter the technical difficulties I ran into. Zac was instrumental in encouraging me along the way and creating a project that I was in the end proud of. There were times in the project when I wanted to give up and switch to a simpler technique, but Zac’s timely help pushed me towards the finish line.

I truly enjoyed figuring out the code for this project since it involved a fair bit of complex logic. I had taken a Python course in one of the previous semesters and my knowledge of coding came in very handy for this project. The next time I will be more vigilant when I use pre-written code because it may not always be correct. I was also surprised that I found soldering extremely therapeutic. From this project, I was able to learn how to make well-soldered connections.

If I have access to resources, I would want to work on the form and the ergonomics of the prototype. I would create a well-designed enclosing to house all the components, I would create more aesthetically pleasing rings that can easily be mounted on the hand of the user. I would also want to test out the prototype with participants in the context of outdoor navigation and evaluate the prototype.

Technical information

Schematic

Schematic for the tactile navigation wearable

Code

/*
   Project 2: Tactile Navigation Wearable
   By Aadya Krishnaprasad

   Description: This is the code for a prototype of a wearable device that assists in navigation by providing tactile directional instructions to the user to assist them in wayfinding while navigating unfamiliar routes in a simple outdoor environment.

   Pin Mapping:

   pin      | mode      | description
   ---------|-----------|-------------
   3          Output      GPS module tx
   4          Input       GPS module rx
   5          Output      Vibration motor (RING1)
   6          Output      Vibration motor (RING2)
   9          Output      Vibration motor (RING3)
   10         Output      Vibration motor (RING4)

*/


// Compass SETUP
#include <QMC5883LCompass.h>
QMC5883LCompass compass;

//Citation: GPS code from: https://create.arduino.cc/projecthub/ruchir1674/how-to-interface-gps-module-neo-6m-with-arduino-8f90ad
// GPS SETUP
#include <SoftwareSerial.h>
#include <TinyGPS.h>
//long   lat,lon; // create variable for latitude and longitude object

float lat = 28.5458, lon = 77.1703; // create variable for latitude and longitude object
//float lat = 40.4412, lon = -79.9435; // Location: Origin, Head West
//float lat = 40.4414, lon = -79.9440; // Location: Waypoint 1, Head North

SoftwareSerial gpsSerial(3, 4); //tx,rx

TinyGPS gps; // create gps object

// RING PIN ASSIGNMENT
const int RING1 = 5;
const int RING2 = 6;
const int RING3 = 7;
const int RING4 = 9;


void setup() {
  Serial.begin(9600);
  compass.init();
  gpsSerial.begin(9600); // connect gps sensor

  pinMode(RING1, OUTPUT);
  pinMode(RING2, OUTPUT);
  pinMode(RING3, OUTPUT);
  pinMode(RING4, OUTPUT);

}


void loop() {

  int a;

  int WAIT = 250;
  long timer = 0;
  if ( (millis() - timer) >= WAIT ) { // millis() is always the number of milliseconds since the Arduino powered up
    // Read compass values ; Compass 0 is East
    compass.read();

    // Return Azimuth reading
    a = compass.getAzimuth();

    Serial.print("A: ");
    Serial.print(a);
    Serial.println();

    timer = millis(); // reset the timer
  }

  // Code to get latitude and longitude coordinates from the GPS
  while (gpsSerial.available()) { // check for gps data
    if (gps.encode(gpsSerial.read())) // encode gps data
    {

      gps.f_get_position(&lat, &lon); // get latitude and longitude

      Serial.print("Position: ");
      Serial.print("Latitude:");
      Serial.print(lat, 6);
      Serial.print(";");
      Serial.print("Longitude:");
      Serial.println(lon, 6);
    }
  }

  String latitude = String(lat, 4);
  String longitude = String(lon, 4);
  Serial.println(latitude + ";" + longitude);



  if ((((40.4411) <= (lat)) && ((lat) <= (40.4415))) && (((-79.9434) >= (lon)) && ((lon) >= (-79.9438)))) {
    Serial.println("Location: Origin, Head West");

    if ((a >= 134) && (a <= 224)) { // West(180)
      goStraight(125);

    } else if ((a >= 316) || (a <= 45)) { // East(0)
      turnAround(250);

    } else if ((a >= 225) && (a <= 315)) { // South(270)
      turnRight100(125);

    } else if ((a >= 46) && (a <= 135)) { // North(90)
      turnLeft100(125);

    }
  }


  if ((((40.4413) <= (lat)) && ((lat) <= (40.4414))) && (((-79.9439) >= (lon)) && ((lon) >= (-79.9441)))) {
    Serial.println("Location: Waypoint 1, Head North");
    // Location: Waypoint 1, Head North

    if ((a >= 46) && (a <= 135)) { // North(90)
      goStraight(125);

    } else if ((a >= 134) && (a <= 224)) { // West(180)
      turnRight100(125);

    } else if ((a >= 225) && (a <= 315)) { // South(270)
      turnAround(250);

    } else if ((a >= 316) || (a <= 45)) { // East(0)
      turnLeft100(125);
    }
  }

  if ((((40.4415) <= (lat)) && ((lat) <= (40.4416))) && (((-79.9439) >= (lon)) && ((lon) >= (-79.9441)))) {
    Serial.println("Location: Waypoint 2, Head West");
    // Location: Waypoint 2, Head West

    if ((a >= 134) && (a <= 224)) { // West(180)
      goStraight(125);

    } else if ((a >= 316) || (a <= 45)) { // East(0)
      turnAround(250);

    } else if ((a >= 225) && (a <= 315)) { // South(270)
      turnRight100(125);

    } else if ((a >= 46) && (a <= 135)) { // North(90)
      turnLeft100(125);

    }
  }

  if ((((40.4416) <= (lat)) && ((lat) <= (40.4417))) && (((-79.9445) >= (lon)) && ((lon) >= (-79.9446)))) {
    Serial.println("Location: Waypoint 3, Head North");
    // Location: Waypoint 3, Head North

    if ((a >= 46) && (a <= 135)) { // North(90)
      goStraight(125);

    } else if ((a >= 134) && (a <= 224)) { // West(180)
      turnRight100(125);

    } else if ((a >= 225) && (a <= 315)) { // South(270)
      turnAround(250);

    } else if ((a >= 316) || (a <= 45)) { // East(0)
      turnLeft100(125);
    }
  }

  if ((((40.4419) <= (lat)) && ((lat) <= (40.4422))) && (((-79.9443) >= (lon)) && ((lon) >= (-79.9445)))) {
    // Location: Destination, Stop
    destinationReached(250);
  }
}

////////////////PATTERNS

// Tactile pattern: If left turn is less than 50 feet away, then vibration motor vibrates at maximum intensity
void turnLeft50(int intensity) {
  analogWrite(RING1, intensity);

}

// Tactile pattern: If left turn is between 50 and 100 feet away, then vibration motor vibrates at medium intensity
void turnLeft100(int intensity) {
  analogWrite(RING2, 0);
  analogWrite(RING3, 0);
  analogWrite(RING4, 0);
  analogWrite(RING1, intensity);

}

void goStraight(int intensity ) {
  analogWrite(RING1, 0);
  analogWrite(RING4, 0);
  analogWrite(RING2, intensity);
  analogWrite(RING3, intensity);
}
 // Tactile pattern: If right turn is less than 50 feet away, then vibration motor vibrates at maximum intensity
void turnRight50(int intensity ) {
  analogWrite(RING2, 0);
  analogWrite(RING3, 0);
  analogWrite(RING1, 0);
  analogWrite(RING4, intensity);

}

// Tactile pattern: If right turn is between 50 and 100 feet away, then vibration motor vibrates at medium intensity
void turnRight100(int intensity ) {
  analogWrite(RING2, 0);
  analogWrite(RING3, 0);
  analogWrite(RING1, 0);
  analogWrite(RING4, intensity);
}

void turnAround(int intensity ) {
  analogWrite(RING1, intensity);
  analogWrite(RING2, intensity);
  analogWrite(RING3, intensity);
  analogWrite(RING4, intensity);
}

void destinationReached(int intensity) {
  analogWrite(RING1, intensity);
  delay(70);
  analogWrite(RING1, 0);
  analogWrite(RING2, intensity);
  delay(70);
  analogWrite(RING2, 0);
  analogWrite(RING3, intensity);
  delay(70);
  analogWrite(RING3, 0);
  analogWrite(RING4, intensity);
  delay(70);
  analogWrite(RING4, 0);
}

For this project, our team was tasked with creating an assistive device for our Osher student Cynthia. The goal was to design a piece of technology to help her in her daily life. We conducted an interview with Cynthia to gain better insight into her life and identify potential problems we could address with our project. Our team decided to design an automated dog trainer to help train Cynthia’s dog Zimi to stop chewing on her sandals and other valuables. We documented our prototypes before moving onto designing our final device.

What We Built

Our device is a dog trainer that senses the presence of an object placed on top of it. If the device senses the object has been moved, alerting that Zimi is chewing on the object, it will trigger a high pitched alarm sound as punishment. The sound is at a frequency that dogs can hear, so as to not disturb Cynthia. There is an additional switch that can turn the alarm on and off to allow Cynthia to remove the object without sounding off the alarm. Other features include a blue light that turns on to indicate the alarm going off, as well as a separate alarm that goes off at 8am to remind Cynthia to take Zimi outside. Over time, the goal of the device is to train Cynthia’s dog to learn what items he should not be chewing  on.

FINAL RenderingS

Labeled components of our device modeled with Fusion 360 .

Overall view of the device: A circular, elevated platform with a wooden base and black glass for the paw print detail.

Detailed view of the device components. The keypad is used to input time and alarm sound settings. The LCD screen displays device modes and user inputs.

Battery door located at bottom of device that can be unscrewed to replace battery power.

Device measurements for scale (mm)

PIR sensors located beneath the black glass. There is a lower surface panel to hold components.

Side view of the device.

TinkerCad INTERACTION

Narrative Sketch

Cynthia is in her kitchen working on her book. Meanwhile, Zimi is walking around the house looking for something to chew on. She sees paper on the platform, her new favorite object to chew on. Zimi walks over and grabs the paper. Suddenly, Zimi hears dog whistles coming from the device. She runs away and the whistling stops. Zimi tries again and the same sound goes off. She gives up and looks for other things to do.

The next morning, Cynthia is eating her breakfast. At 8AM, she hears the device going off and sees the LCD message reminding her to walk Zimi. It was too loud of a sound in the morning, so she turns down the volume. She turns off the alarm, finishes her breakfast, and brings Zimi outside.

In the afternoon, Cynthia resumes in editing her book. One of her papers fell to the ground without her realizing it. Zimi sees the paper, but decides not to bother with it.

How We Got Here

The process of creating this device was definitely a learning experience for us as we had to overcome various challenges along the way. The initial sketch for the device gave us a solid starting point to work off of, but we later made significant changes to the design for both aesthetic and functional purposes. We cycled through iterations of the device, adding new components such as an 8am alarm and volume adjustment after receiving feedback from Cynthia. In our work plan, we scheduled to front load most of the work, which meant finishing the circuits and model renderings within a week. However, we ended up diverging from our original schedule due to last minute revisions. 

Initial concept sketches.

One major setback was that TinkerCad doesn’t have the Real Time Clock part, so we had to improvise by creating an alternate means of tracking time through a keypad as the user interface to set the time and then having the device calculate current time from that input. As a result, we got rid of the potentiometer which we initially planned to have control volume since we could just incorporate volume settings onto the keypad. 

These changes to the circuit also meant changes to the physical model design. This posed another challenge since we did not anticipate these changes and had to consider ways to add this new keypad component which was quite large and clunky in appearance. We considered adding a separate platform to fit the keypad, but it was not as clean design-wise. In the end, we resized and rearranged the new components to all fit on the top surface which ended up looking alright. 

Added potentiometer to adjust alarm volume which was later replaced with keypad .

Rendering the models was time-consuming due to the fact that the renders were made too soon without accounting for last minute changes which required re-rendering. In addition, the quality of the renderings were turning out blurry, and it was only until much later that we discovered it was because the image dimensions were set to very small values, making them pixelated. After increasing image size to 3840px by 2160px, the quality improved dramatically, and we were finally able to get decent renderings of the finished device.

Exported cloud renderings which had to be re-rendered to match updated design.

Blurry rendering of unfinished model.

Render settings menu with the default image width and height set to a small pixel size.

Conclusion

We received a mix of feedback during the presentation of our final device. Overall, people liked the aesthetics of our project and the concept behind it. However, some expressed concerns if this form of positive punishment will be successful in real life, which was also a concern we had while developing the device. We mainly worked based on a theory and ran with it, but understood that such a device would have to be tested on Zimi to see if it will actually be useful, though such testing would take a while to get any conclusive results since our device hopes to affect behavior. Others had more specific concerns such as:

Is the alarm for Cynthia with the unit? Would she be able to hear it if it is placed on top of her shoe?

Would this only work for 1 pair of shoes? Would Cynthia need multiple of the devices? For all her shoes?

In regards to the audibility issue, we positioned the speaker away from the middle of the device and in the corner with all of the other electronic inputs/outputs such as the keypad and LED. The device was built large enough that this area would not be covered by a shoe for example. Additionally, Cynthia wanted the ability to change the volume of the device so she can increase/decrease the volume depending on what room she places the device in so she can reasonably hear it. But, we agreed that we can continue to optimize the positioning of the speaker on the device and even the type of speaker. A lot of the feedback was concerned on the practicality of the device’s application with a shoe. We used shoes as an example of what Zimi likes to chew on, but we hope that this device has the versatility to train Zimi not to chew on a variety of small objects. Ideally, we hope Zimi will associate the objects placed on the device with punishment and bad things she should not be chewing on.

After our first prototype, Cynthia expressed Zimi chewing on shoes was no longer a problem since Cynthia decided to hide all of her shoes. However, we agreed hiding shoes might not be practical in the long run as we wanted to teach and train Zimi as a more long-lasting solution to her chewing habits. Cynthia expressed concerns that Zimi is starting to chew on paper she finds on the floor. If our device was built, we would first try it with paper and see how her behavior changes. If successful and Zimi finds other things to chew, then we would potentially put whatever new object Zimi likes to chew onto the device so that we can prime her not to chew on those types of items.

Collaborating and developing a device remotely had its challenges. Due to online learning, we shifted to using mainly CAD and TinkerCad to develop and present our work. A lot of the project work was split into individual tasks which we worked separately on. Some tasks were more difficult to collaborate on remotely, for instance creating the CAD model or wiring the circuit. This task allocation actually turned out to work pretty well since we were able to concentrate on our own side of the project. In the end, although we each addressed separate project domains, we were able to create a cohesive device by combining our components.

Reflecting on the process of building this device, one of the biggest challenges was using TinkerCad, which is slow, has limited parts, and definitely buggy. For instance, we originally wanted to use an IR remote instead of a keypad. However, the library for the IR remote could not be added to the code without creating errors. Additionally, simulating the interaction is not the same as real life. The project would have definitely been easier and more constructive if we have the electronic parts with us to physically troubleshoot challenges in the wiring and code. If we were to do the project again, we would like to do this part of the project physically in real life, so that we can have a better understanding of the functionality of the device and better improve it.

The CAD portion also had it has its own challenges, but in the end improved the quality of the overall project. For future CAD models, we learned it’s important to leave room for error and design adjustment before moving onto final rendering. We would have had to use CAD anyway, but since we didn’t have to fabricate it, it expanded the potential of the overall physical device. Our specific device can be fabricated, but the situation allowed us to design more complex designs without being held back by fabrication difficulties.

Working with an older person and designing a device specifically for them was a new experience for us. We thought it was a wonderful experience learning about Cynthia during our initial interview with her. She has lived in Pittsburgh for a long time so she was able to share some insights about her favorite places to eat and drink. One of the major takeaways that became apparently clear especially after the final presentations were that many of the devices built address issues that many people people might have problems with and have the potential to reach a wider population.

Technical Details

TINKERCAD

TinkerCAD (https://www.tinkercad.com/things/91w05kUAJCs)

Breadboard image of Tinkercad model

Code

#include <Adafruit_NeoPixel.h>
#include <Keypad.h>
#include <LiquidCrystal.h>

/** 
   Title: Dog Trainer
   Author: Gracia Genero
   Description: This code controls the components listed below. 
   It fires Zimi's alarm through the speaker/LED when motion is detected
   through the PIR Sensor. The PIR sensor and Zimi's alarm can be deactivated
   by flipping the switch. Additionally, this device keeps track of time'
   time, but requires the user to input a current military time to do so accurately. 
   At 8AM a separate alarm goes off to remind the user (Cynthia) to walk her dog
   Zimi. This alarm can be turned off through the keypad input. The volume of 
   Cynthia's alarm can also be adjusted through the keypad. The LCD screen displays messages
   when the alarm is activated or when an input is in progress of being typed into the
   keypad. Otherwise, it displays the calculated current time and volume. 
   
  
   Pin mapping:
     
   pin        | mode   | description
  ------------|--------|------------------------------------
   A3         | input  | PIR motion sensor
   A4         | output | NeoPixel Ring 16
   A5         | output | speaker
   A2         | input  | switch
   A0,A1,11,10| input  | keypad - row pins
   8,7,6,5    | input  | keypad - column pins
   4,3,9,2    | input  | LCD - DB4-DB7
   13         | input  | LCD - register select pin
   12         | input  | LCD - enable pin
   

  Credits:
  https://www.tinkercad.com/things/ascn1ro2gFR-lcd-display
  https://www.tinkercad.com/things/jLFSyDIIEpj-pir-motion-sensor-with-arduino-blocks
  https://www.tinkercad.com/things/k4BAT8c26NN-print-4x4-keypad-input-to-lcd
 
  
 **/

const int SPEAKERPIN = A5;
const int SWITCHPIN = A2;
const int LEDPIN = A4;
const int LEDCOUNT = 16;       // number of NeoPixels in series
const int PIRPIN = A3; 
int pirState = LOW;             // we start, assuming no motion detected
int pirVal = 0;                 // variable for reading the pin status
int ledState = LOW;
int piezState = LOW;
String vol = "5"; // range 0-9
String setTime = "0500";
String keypadInput = "";
String alarmTime = "0800";
String currentTime = "0500";
int CynthiaAlarm = 0;
int turnedOff = 0;

int editMode = 0; 
int volMode = 0;
unsigned long int setMillis= 0; 
int LCDCol = 0;

const byte ROWS = 4; //four rows
const byte COLS = 4; //three columns
char keys[ROWS][COLS] = {
  {'1','2','3','A'},
  {'4','5','6','B'},
  {'7','8','9','C'},
  {'*','0','#','D'}
};
byte rowPins[ROWS] = {A0, A1, 11, 10}; //connect to the row pinouts of the keypad
byte colPins[COLS] = {8, 7, 6, 5}; //connect to the column pinouts of the keypad
Keypad keypad = Keypad( makeKeymap(keys), rowPins, colPins, ROWS, COLS );

// Object
Adafruit_NeoPixel rgb(LEDCOUNT, LEDPIN, NEO_RGB + NEO_KHZ800);
LiquidCrystal lcd(13, 12, 4, 3, 9, 2);



void setup() {
  pinMode(SWITCHPIN, INPUT);
  pinMode(LEDPIN, OUTPUT);      // declare LED as output
  pinMode(PIRPIN, INPUT);     // declare sensor as input
  pinMode(SPEAKERPIN, OUTPUT);
  
  lcd.begin(16, 2);
  Serial.begin(9600);

}

void loop(){
  
  char key = keypad.getKey();
  if (key){
    Serial.println(key);
    
    if (key == '*') { 
    setTime = keypadInput; 
    setMillis = millis(); 
    Serial.println("Set time = " + String(setTime) + " at millis "+ String(setMillis));
    editMode = 0;
    keypadInput = "";
    Serial.println("Time Set");
    LCDCol = 0; 
    lcd.clear();
    }
    
    if (editMode == 1) {
      keypadInput += String(key);
      lcd.setCursor(LCDCol, 1);
      lcd.print(key);
      LCDCol += 1;
    }

    if (volMode == 1) { 
      vol = String(key);
      volMode = 0; 
       }
    
    if (key == 'A') { 
      editMode = 1;
      Serial.println("Setting Current Time");
      lcd.clear();
      lcd.setCursor(0,0);
      lcd.print("Set current time");
 
    }
    
    if (key == 'B') { 
      volMode = 1;
      Serial.println("Changing Volume");
      lcd.clear();
      lcd.setCursor(0,0);
      lcd.print("Set volume");
    }
  }
  
  Serial.println("Volume = " + vol);
  
  // Calculates current time based on last current time by the user
  int deltaMin = ((millis() - setMillis) / 60000) % 60;
  int deltaHour = ((millis() - setMillis) / 60000) / 60; 
  
  String setMin = setTime.substring(2);
  int min = setMin.toInt();
  String setHour = setTime.substring(0,2); 
  int hour = setHour.toInt();
  
  int newMin = (min + deltaMin) % 60;
  String newMinStr = String(newMin);
  if (((newMin % 10) >= 0) and ((newMin / 10) < 1)) {
  	newMinStr = "0" + newMinStr; 
  }
  if (((min + deltaMin) / 60) >= 1) { deltaHour += 1; }
  int newHr = (hour + deltaHour) % 24;
  String newHrStr = String(newHr);
  if (((newHr % 10) >= 0) and ((newHr / 10) < 1)) {
  	newHrStr = "0" + newHrStr; 
  }
  currentTime = newHrStr + newMinStr;
  Serial.println("SetTime = " + setTime + ", Current Time = " + currentTime);
  
  if (((editMode == 0) and (volMode == 0)) and (CynthiaAlarm == 0)) {
      lcd.setCursor(0, 0);
      lcd.print("Time: " + currentTime);
      lcd.setCursor(0, 1);
      lcd.print("Volume: " + vol);
    }
  
  int switchState = digitalRead(SWITCHPIN);
  pirVal = digitalRead(PIRPIN);  // read input value
  Serial.println("PIRSensor = " + String(pirVal));
  
  if ((currentTime == alarmTime) and (turnedOff == 0)) {
    CynthiaAlarm = 1; 
    }
  if (currentTime.toInt() >= 803) {
    CynthiaAlarm = 0;
    turnedOff = 0;
    lcd.clear();
  }
  if ((key == 'C') and (currentTime.toInt() <= 803) and (currentTime.toInt() >= 800)) {
  	turnedOff = 1;
    CynthiaAlarm = 0;
    lcd.clear();
  }
  if (CynthiaAlarm == 1) {
      tone(SPEAKERPIN, 100);
      lcd.clear();
      lcd.print("It's time for");
      lcd.setCursor(0,1);
      lcd.print("a walk!"); 
  }
  else {
  if (switchState == LOW) {
  if (pirVal == HIGH) {          // check if the input is HIGH
    // turn on LED blue
    for(int i=0; i<rgb.numPixels(); i++) {
    rgb.setPixelColor(i, 0, 0, 150);    // sets the RGB values for pixel number i  
    // rgb.show();
    }  
    rgb.show();
    analogWrite(SPEAKERPIN, vol.toInt()*14); // sets duty cycle or volume of piezo buzzer 
    tone(SPEAKERPIN, 25000); // speaker sends off sound frequencies only dogs can hear
    if (pirState == LOW) {
      Serial.println("Motion detected");
      pirState = HIGH;
    }
  } 
  else {
    noTone(SPEAKERPIN); // turns off speaker
    rgb.clear(); // turn off LED
    rgb.show(); 
    if (pirState == HIGH){
      Serial.println("Motion ended");
      pirState = LOW;
    }
  
  }
  }
    
  else {
  
  noTone(SPEAKERPIN);
  rgb.clear(); // turn off LED
  rgb.show(); 
  }
  }
}

SCHEMATIC

Schematic of dog trainer device

Design Files

Dog Trainer Design Files (Fusion 360)

For the final project of Physical Computing(60-223) course at Carnegie Mellon University, each group of three CMU students was assigned to an Osher student to build an assistive device for the older adults. Our group had the fortune to work with an amazing Osher student who herself is a CMU alum, Emily Eckel. Through multiple stages of interviews, ideations, discussion, and critiques, we decided to create an entertaining ‘Mouse Detector’ that can detect mouse at her house so that she can bring her son’s cat from upstairs. The two most important goals of this project were functionality and entertainment. The device should be able to detect a mouse, nothing else, with high accuracy, and be able to notify Emily. On top of that, due to the COVID-19 pandemic situation, Emily had to spend more time at her house which makes her a bit depressed; thus she wanted a device that can give her small joy in daily life.

Although this project was initially planned to have an actual physical prototype in the end, because of the COVID-19 pandemic situation where all CMU courses have officially conducted remotely, we had no choice but to make this project virtual. To provide a more constructive and tangible overview of our idea to Emily and everyone in this process, we decided to make a physical crafted prototype, a CAD design simulation, and a circuit/technical prototype. For your reference, more details about the initial ideation and plans can be found here, and the initial model and details can be found here.

What We Built

We built a mouse detector, with two parts: the sensor unit which detects if a mouse is nearby, and the control panel unit that notifies Emily if a mouse has been detected and allows her to control the notifications. When a mouse is detected, the control panel lights up, and a small door opens and a mouse sign emerges. If it is daytime, music from Mickey Mouse plays. The volume knob controls the loudness of the music, and the brightness knob controls the brightness of the panel when it lights up. When the reset button is pressed, the mouse sign goes back in and the door closes, and light and music turns off.

Final appearance rendering

Overall image (character panel going back in after device has been reset)

Volume and brightness control knobs

Speakers

Notification on

Tinkercad Interaction

Physical MOCKUP

A front view of the controller unit crafted prototype (two green and red knobs each are volume and brightness controller, a door and a mouse panel is a cuckoo bird clock-like notification, and a yellow rectangular button is a reset button)

A top view of the controller unit crafted prototype (two yellow circles are a speaker unit for notifications)

A side view of the controller unit crafted prototype

A back view of the controller unit crafted prototype (magnets for the convenience of installing it on the fridge)

A video of how the cuckoo bird clock-like notification works like

A front view of the sensor unit crafted prototype (cheese shape for entertainment)

A back view of the sensor unit crafted prototype (a plug is directed connected to the unit)

A side view of the sensor unit crafted prototype (blue and red pins represent IR proximity sensors that will be installed toward the fridge)

How the controller unit would look like when it is installed on the fridge

How the sensor unit looks like when it is plugged into an outlet (sensors are facing toward the fridge where Emily said mouse appear the most at her house)

NARRATIVE SKETCH

While Emily is at home, a mouse appears from behind the fridge, as usual. As soon as the detector that is plugged into the outlet by the fridge detects the moving mouse, the control panel above on the side of the fridge glows. The fun music plays, and the mouse character pops out of the small doors on the control panel. The sound wouldn’t go off at night, but since it’s daytime and Emily is awake, she needs to come to the fridge and press the reset button to stop the notification. She then calls her son to come into the house and for his cat to catch the mouse as soon as possible.

How We Got Here

Our original schedule

When we were finalizing mechanism methods and components, we took some more time than we had planned in our work plan. This was because when we made the original schedule, we did not anticipate making such a big change in the final product. Although we spent more time in brainstorming and revising out design, the schedule worked out smoothly in the end, it took us less time to build the actual prototypes because the discussion we had earlier was very detailed.

Our prototype was a single-component mouse detector near the ground, with both the detection, control and notification functionalities. However, although a single-component device might be easy to use, we realized having both controller knobs and reset button near the floor would be inconvenient and dangerous for Emily whose back hurts when she bends her back and stands up. On top of that, we found it more visible to have notifications on our eye level rather than having it on the floor, and especially near the fridge where Emily consistently passes by and uses. After our in-depth discussion about the project, we decided to have two components—a sensor unit near the floor and a controller unit on the fridge—in our final product. We first took some time to figure out how they would be powered, and also how they would be connected to each other. Later we essentially redesigned our 3D models, and later built a physical mockup and a corresponding Tinkercad circuit.

A front overview of the digital rendering draft

A back overview of the digital rendering draft

For designing the appearance and creating digital renderings, the main components within the control panel unit were kept as they were developed early on, including 2 speakers, 2 knobs, a pair of doors, 1 character panel,  1 reset button, and magnets on the back side. Since the very first iterations, the form was modified to give a soft and fun look, and knobs were made taller for better usability. The number of magnets was increased for them to be distributed more evenly and for sturdiness.

A front overview of the digital rendering draft with material details added

Then, we changed from a simple opaque material to a yellow, semi-transparent material to make it seem less serene or too much like a common house appliance, and more fun, as well as being able to transmit light when the notification light turns on. Towards the end, we learned that it actually takes a very long time to render transparent material, so we had some difficulty in producing high-quality light-on renders. We could have allocated more time to make this part go more smoothly.

We also used metal to clearly distinguish knobs to be interacted with and the bottom unit, which would contain any internal parts. We also added texture (texts layered on the front surface) to indicate what the knobs are and how to use them.

Electronically, we decided to power the detector part by plugging in to the outlet, and the control panel part by a USB cable from the outlet, after the hearing from Emily that she wanted them to powered by wall power. The two parts would wirelessly communicate through radio transceivers attached to the two Arduinos. However, since Tinkercad does not have the radio transceiver part, in our Tinkercad simulation the two Arduinos are directly connected by wires with two I/O pins, since the communication was simple enough. Another change from our original prototype was changing the sensor component from a PIR sensor to an IR proximity sensor, after feedbacks from our instructor Zach. Since Tinkercad does not have the part, we used an ambient light sensor to model the IR proximity sensor component.

The interior of the sensor unit. There are multiple head pins which the end are hot-glued

For crafting, the hardest part was deciding which materials to use. Because we  are not testing the whether the device actually works, the most important aspect of these materials were how easy it is to craft, and how close can it reflect our design. Due to the pandemic, there were some limitations in getting materials. We used corrugated cardboards for the general structure, stamp-looking fridge magnets for controller knobs, and two different colors of plastic head push pins which each represents IR receiver and IR emitter LED. However, using push pins were quite dangerous because the end parts are very sharp. For safety reasons only, we needed to hot-glue the end part although these details were not necessary.

The connection between the appropriate size plug and the sensor unit

On top of that, there were some difficulties finding an appropriate size and weight of a plug connected to the sensor unit. If we had a chance to implement our design in the lab, this would not have be an issue; however, as we used a cardboard to make the body structure of the sensor unit, when the plug was too big and heavy, the structure collapsed. After multiple candidates, we found the plug that fits the sensor unit size and also not too heavy so that the craft can maintain its shape.

4. Conclusions and lessons Learned

During the final critique, we got multiple positive feedbacks about dividing the device into two parts, which was very rewarding because brainstorming and refining the idea was one of the biggest challenges in our project.

However, some feedback raised the concern about how accurate the sensor unit will be. Given the current situation, it is difficult to physically test, but we could have done some more research to predict better what the optimal angle or area size would be for the specific type and number of detectors. Moreover, in the normal lab setting, it would be hard to test it actually with mouse present; thus, if we were to test it in the lab, we would need to come up with a valid alternative testing method.

One feedback we got was that the novelty and entertainment components of this product might wear out. We think that this is definitely a valid feedback, but we think adding the fun part (the “cuckoo clock” notification) makes designing the product a bit more interesting compared to a simpler design. If we had more time, adding a function to change the alarm sound would make interacting with the product more interesting, and avoids the possibility that the user get annoyed by the alarm sound after using it for a while.

On top of that, during the feedback session with the final crafted prototype, there was a feedback that it would be better to have an on/off switch and a switch that can manually set the day and night mode. We had a discussion about adding such features, but ended up concluding that it is more important to make the device more concise and small as Emily requested it to be than to add additional features. However, there was a feedback during the final presentation suggesting an idea or merging volume and brightness knobs into one, which we have not thought of but seems very interesting and doable. If we have further opportunities to develop this device, we would like to further discuss about this point.

Regarding the teamwork, because our team members were working remotely from each other in different countries, it was difficult to share progress and iterations, and communication was often rather simplified. This made it a little harder to put together ideas and developments, so we decided that it would be the best if each person having a focus, and the process of discussing as a group first, building our own parts/prototypes, discussing our individual decisions and potential changes, and revising our parts. This worked fairly well as we could each take an ample amount of time to develop diverse prototypes, and take good feedback from each other.

Working with a client itself was definitely new to us but very valuable. Unlike following instructions or making a device for myself which were two prior projects conducted during the course, making a device for a person that I have never met before taught me how to interact with clients and how to get to know the client personally. Most importantly, we realized there were many considerations and assumptions we made without thorough decision making processes when we were doing a project on ourselves. For example, it would have been obvious to make the device installed on the desk without any question even just by imagining how we would use it, because that is where we go often; but it was not for Emily. We learned how to interview clients and to what extent we should be interviewing the client in terms of details, and in many cases interviewing the client life pattern turns out to be useful although it does not seem to be related to the project in the beginning.

We were so fortunate to have Emily, who is not only so warm and nice but also was very engaged in project such by giving objective feedbacks and her own ideas, as our client. She was also so opened in giving many small details about her life in general, which helped us a lot in constructively building the model. It truly made the virtual process a lot easier and more productive. However, we could have met her a couple more times to ask her for feedback and take some opportunities to gain more insights, reactions, and any recommendations. During this process, we also learned a little bit about what are some things to consider when designing for older people, such as the terms we use to describe their actions or needs, or keeping in mind their physical capabilities and limitations.

Technical Details

Tinkercad Link

Link

Breadboard Image

Code for the sensor unit’s Arduino

/*
 * Final Project: Mouse Detector for Emily
 * Youie Cho, Yun Lee, Alan Zhu
 *
 * Description: This is the code that would be on the sensor unit's
 * Arduino. Essentially it reads an input of a IR Proximity sensor
 * (modeled by a phototransistor in Tinkercad) to check if a mouse
 * is nearby. If a mouse is detected, it sends a digital signal to
 * the control unit's Arduino. It also reads a photoresistor to
 * identify if it is daytime, and sends the digial signal to the
 * control unit.
 * 
 * Pin mapping:
 * pin   | mode   | description
 * ------|--------|------------
 * A0     input    phototransistor (modeling a IR Proximity sensor)
 * A1     input    photoresistor
 * 12     output   direct digital output to the control unit  
 * 13     output   direct digital output to the control unit
 * 
 */

//input pins
const int PHOTOTRANSISTOR_PIN = A1; 
const int PHOTORESISTOR_PIN = A0; 

//output pins, both send signals to the control panel arduino
const int MOUSE_PIN = 13;
const int DAYTIME_PIN = 12;

int lightLimit = 650;
int mouseLimit = 900;
int lightVal, mouseVal;

void setup()
{
  Serial.begin(9600);
  pinMode(PHOTOTRANSISTOR_PIN, INPUT);
  pinMode(PHOTORESISTOR_PIN, INPUT);
  pinMode(MOUSE_PIN, OUTPUT);
  pinMode(DAYTIME_PIN, OUTPUT);
}

void loop()
{
  //Reads photoresistor, if the readings exceed the preset limit,
  //it is daytime. Sends corresponding digital signal to the control
  //panel Arduino.
  lightVal = analogRead(PHOTORESISTOR_PIN); 
  if(lightVal >= lightLimit) digitalWrite(DAYTIME_PIN, HIGH);
  else digitalWrite(DAYTIME_PIN, LOW);
  
  //Reads phototransistor value (modeling IR Proximity Sensor),
  //if exceeds preset limit, mouse is detected. Sends corresponding
  //digital signal to the control panel Arduino.
  mouseVal = analogRead(PHOTOTRANSISTOR_PIN); 
  if(mouseVal >= mouseLimit) digitalWrite(MOUSE_PIN, HIGH);
  else digitalWrite(MOUSE_PIN, LOW);
  
  delay(150);
}

Code for the control panel unit’s Arduino

/*
 * Final Project: Mouse Detector for Emily
 * Youie Cho, Yun Lee, Alan Zhu
 *
 * Description: This is the code that would be on the control panel
 * unit's Arduino. It reads the inputs from the sensor unit's Arduino
 * of whether a mouse has been detected and if it is daytime. If a mouse
 * is detected, the LED lights up, two servomotors rotate to open a
 * door. The speaker plays music in addition if it is daytime. The brightness
 * knob potentiometer sets the brightness level for the LED, and the volume
 * knob sets the loudness of the speaker. If the reset button is pressed,
 * the LED turns off, two servomotors return to its original position and
 * closes the door, and the music is turned off.
 * 
 * Pin mapping:
 * pin   | mode   | description
 * ------|--------|------------
 * A1     input    potentiometer (brightness knob)
 * A3     output   LED
 * A4     output   servo 1  
 * A5     output   servo 2
 * 3	  input    reset button
 * 7      output   piezo speaker
 * 12     input    direct digital input from the sensor unit Arduino
 * 13     input    direct digital input from the sensor unit Arduino
 *
 */

#include <Servo.h>

Servo servo1, servo2;

//input pins
const int MOUSE_PIN = 13; // from the other Arduino
const int DAYTIME_PIN = 12; // from the other Arduino
const int LIGHT_POT_PIN = A1;
const int BUTTON_PIN = 3;

//output pins
const int PIEZO_PIN = 7;
const int LED_PIN = A3;
const int SERVO1_PIN = A4;
const int SERVO2_PIN = A5;

int photoLimit, photoVal, lightVal;
bool dayTime, mouse, button;

void setup()
{
  Serial.begin(9600);
  servo1.attach(SERVO1_PIN);
  servo1.write(0);
  servo2.attach(SERVO2_PIN);
  servo2.write(180);
  pinMode(MOUSE_PIN, INPUT);
  pinMode(DAYTIME_PIN, INPUT);
  pinMode(LIGHT_POT_PIN, INPUT);
  pinMode(BUTTON_PIN, INPUT_PULLUP);
  pinMode(PIEZO_PIN, OUTPUT);
  pinMode(LED_PIN, OUTPUT);
  
}

void loop()
{ 
  //read the brightness knob to set brigtness level for LED
  lightVal = map(analogRead(LIGHT_POT_PIN), 0, 1023, 0, 255);
  
  //receive from the mouse detector arduino if mouse is detected
  mouse = digitalRead(MOUSE_PIN);
  
  //receive from the mouse detector arduino if it is daytime
  dayTime = digitalRead(DAYTIME_PIN);
  
  //read if reset button is pressed
  button = digitalRead(BUTTON_PIN);
  
  //If mouse is detected, turn on light
  //and turn the two servos to open the doors
  if(mouse) {
    analogWrite(LED_PIN, lightVal);
    servo1.write(45);
    servo2.write(135);
  	Serial.println("Mouse detected!!");
    if(dayTime) playMusic(); //only play music if it is daytime
  }
  
  //If reset button pressed, turn off light and music,
  //and reset the servos back to its original position
  if(!button) {
    analogWrite(LED_PIN, 0);
    noTone(PIEZO_PIN);
    servo1.write(0);
    servo2.write(180);
    mouse = false;
    Serial.println("Reset");
  }
  
  delay(100);
}

//play a tune on the speaker 
void playMusic()
{
  tone(PIEZO_PIN, 600, 150);
  delay(150);
  tone(PIEZO_PIN, 800, 150);
  delay(150);
  tone(PIEZO_PIN, 1000, 150);
  delay(150);
  tone(PIEZO_PIN, 1200, 150);
  delay(350);
}

Schematic

Design Files

mouse_CAD

Team Beth Prototype Documentation

What we built

Our moss spreading device is a simple design. It is a remote-controlled car with a blender on the top, a tank to hold moss, and a funnel to dispense the mixture at the bottom. The purpose of the car is to cover land with a beautiful, even layer of moss with minimal effort.

Car Model

isometric view

side view

top view

front view

Controller Model

Controller Exploded View

Controller top view

TinkerCad

Each button sends a signal to the IR receiver, represented by the remote in Tinkercad.

(Subtitles may be covered by the bottom bar)

Movement demo:

Blend/dispense demo:

After some time, Beth sees that her yard is looking a little dull so she feels like beautifying it. Some of it is covered in moss and some is not. She does some research and finds that she can make moss by blending buttermilk and moss. The only problem is that her yard is extremely huge and complex with random patches of open area mixed with sporadic trees. It’s not convenient to only have one blender-full of moss at a time to cover 3 acres of land.

She grabs her remote controlled robot, places it in her yard and flips the power toggle to the on position. Beth begins to fill up the car’s tank by blending her moss and buttermilk with the blend button on the remote. After two full blenders, Beth starts to drive the robot by pressing the directional buttons on the remote and also pressing the dispense button so she could spread the moss mix however she would like.

How we got here

Vehicle design process

Here is our original idea for moving the fluid in the blender to the tank. It consisted of have a servo move a thin disk in and out of the nozzles. The servo would move the disk in while we didn’t need any fluid to be transferred. It would move the disk out of the way to let fluid through. We very slowly (but surely) realized that this solution wouldn’t work quite well due to the placement of the motor and how thin the disk would have to be.

Brainstorming using servomotor to open blender to tank

We then came up with a slightly better solution that would allow for easier placement of the servo and less spillage.

However, a problem we encountered right after was figuring out how to dispense the fluid that was transferred into the tank onto the ground without a pump. Here is picture showing a picture of our primary tank shape, which was close to a rectangular prism.

rectangular shaped tank

We were already trying to build to minimize weight which meant adding another motor for a pump would be costly. We then decided to sacrifice some efficiency with holding fluid and make the tank angled so that gravity could push out the fluid.

drawings for gravity powered tank

final tank design

One big problem we had to address was the dispensing mechanism.  We had to take into account leaking a mixture of thick-milkshake-like consistency, the weight of the liquid, and space allocation in the robot. A few ideas we had was a way to use magnets to open and close a door, gears that would use DC motors to bring a door up and down (like a garage door), and ways to use servos to pull a plane up and down. However, none of these ideas really solved the problem of massive waste while opening and closing.

Drawbridge style idea

Our solution was to mimic the blender to tank design from earlier and spread the liquid with a funnel-shaped output.

funnel design for spread

The biggest breakthrough decision we made was to focus on moss dispensing; our original idea was to also attach a cordless leaf blower. The weight of a strong enough leaf blower added on to the weight of the battery and all the liquid in the tank would make the robot need a lot of power and super strong motors. Beth shared this concern and pointed us in the right direction.

Our very first design, original plan to have leaf blowing moss robot

moss dispenser only iteration

Controller design process

The controller was initially designed to be held with both hand, inspired by Nintendo controller.

first controller iteration

However, after the feedback from Beth that she needs to spare a hand for other tasks, we changed the design to one-hand controller and made sure that the form can fit nicely in one hand.

second controller iteration

Work ethic and pace

Here is a schedule we had set for each other to get all the components of the project done.

set work schedule

We all stayed on track and it was because we spread out the tasks evenly and keep in mind what would take longer. It also helps that we made our tasks modular. Once one task was done it served as a bolster to completing the following task.

Conclusions and lessons learned

A question was brought up about the cleaning of this device: “If there is buttermilk left inside then there will be mold forming inside after a while. Will that affect future usage of the machine?” To clean the device, Beth would fill up the blender with water, blend, and let the water run through the tank. She could run water through until the dispensed water is clean.

Another comment was: “Have you thought about making the car solar power?” This is a great idea. To not add to the weight of the robot itself, we can make a solar powered charging station. But since Beth said she doesn’t get much sun, we would still have the AC adapter option.

A lot of comments were about testing:

“When the blender is full is the device top heavy and unstable?”

“How are you ensuring that your motors will be able to carry the weight of the entire device + when it’s full?”

“This is a clever design, however I worry about the vibration of the blender and how that will impact other components of your moss rover.”

The vibration of the blender is actually something that we didn’t think about. It is a valid concern that would require testing to address. We think it was unfortunate that we couldn’t test these problems but if we find the right parts, like a strong battery and DC motors, our design would be effective. If we build up a physical prototype, Beth said we can head out to her yard and test out the device and we can answer these questions then. Overall, working with Beth was a lot of fun. It was cool that we were able to build a device for a very unique problem.

We had several Zoom meetings to figure out the mechanism together (as shown in all those sketchy sketches of the robot model above). These meetings were very fruitful and effective in coming up with theoretical solutions for some technical difficulties, such as how to design the stoppers and funnels. However, because we were working remotely, we didn’t have the opportunity to test out the area our tank could cover and how effective different batteries, motors, blenders, and wheels would be. Also we were unable to test the stoppers between the blender and tank and the funnel which are important parts of our design. Also because we couldn’t meet, we divided the remote work quite clearly and were each responsible for a certain part. In retrospect, this was effective but overall, communication, about the form, electronics, and prototyping, would have been much easier in person.

We learned how to balance ambition with practicality during our prototyping process. Our first iteration, the leaf blowing moss dispenser, was a really clever idea to us, but Beth was concerned about the practicality. So for our second iteration, we made our scope more realistic. In retrospect, we should have made our process more modular from the start. If we broke down our ideas into smaller components in the beginning and made a Gantt-chart-like plan, we would have been able to see how realistic it actually was.

The biggest takeaway from our experience working with an elderly person was that checking in with Beth during our second iteration prototyping process might have helped improve our design. As we approached the end of the project, we got very invested in how the mechanism works for the robot since we were dealing with a fairly complex system. While such investigation is very valuable and we each learned a lot from the prototyping process, we put aside the need of Beth, who was supposed to be the main focus of our design. From our previous conversations with Beth on our first idea, she pointed out how some actual circumstances were different from our assumptions so we needed to take into those real-life factors into considerations, which was one of the biggest drive for us to shift to our second iteration, making our machine less ambitious and more simplistic and functional. Similarly, it would be nice if we talk to Beth to get some feedback for our second iteration, which might push our project to an even more practical, more human-centered design result.

Technical details

Robot electronics:

Breadboard image robot

Remote electronics:

https://www.tinkercad.com/things/4zUE3ML0yFY

Breadboard image remote

RC car schematic

Remote schematic

#include <IRremote.h>
#include <Servo.h>

/* Beth's Moss Dispenser
 *  
 *  Description: A remote controlled moss blender and dispenser to help Beth with her
 *  yard beautifying processes.
 *  
 *  https://www.tinkercad.com/things/1E1FraSAW9K
 *  
 * Pin mapping:
 * 
 * pin   | mode   | description
 * ------|--------|------------
 * 11     input     IR receiver   
 * 3      output    Left Front Motor Input 1  
 * 4      output    Left Front Motor Input 2
 * 5      output    Right Front Motor Input 1  
 * 6      output    Right Front Motor Input 2 
 * 7      output    Left Back Motor Input 1  
 * 8      output    Left Back Motor Input 2
 * 9      output    Right Back Motor Input 1  
 * 10     output    Right Back Motor Input 2 
 * A5     output    Blender to tank servomotor
 * A4     output    Dispensing door servomotor
 * A3     output    Blender Motor Input 1
 * A2     output    Blender Motor Input 2
 * 
 * 
 * Credit: https://www.instructables.com/id/Arduino-Modules-L298N-Dual-H-Bridge-Motor-Controll/
 * 
*/


const int RECV_PIN = 11;
const int LFIN1 = 3;
const int LFIN2 = 4;
const int RFIN1 = 5;
const int RFIN2 = 6;
const int LBIN1 = 7;
const int LBIN2 = 8;
const int RBIN1 = 9;
const int RBIN2 = 10;
const int BLEND2TK = A5;
const int DOOR = A4;
const int BLENDIN1 = 12;
const int BLENDIN2 = 13;


Servo blenderservo;
Servo doorservo;
unsigned int blendtimer = -20000;

unsigned int lastPress = 0;

bool dispensing = false;

bool on = true;

//IR Library stuff
IRrecv irrecv(RECV_PIN);
decode_results results;

void setup()
{
  pinMode(LFIN1, OUTPUT);
  pinMode(LFIN2, OUTPUT);
  pinMode(RFIN1, OUTPUT);
  pinMode(RFIN2, OUTPUT);
  pinMode(LBIN1, OUTPUT);
  pinMode(LBIN2, OUTPUT);
  pinMode(RBIN1, OUTPUT);
  pinMode(RBIN2, OUTPUT);
  pinMode(BLENDIN1, OUTPUT);
  pinMode(BLENDIN2, OUTPUT);

  blenderservo.attach(BLEND2TK);
  doorservo.attach(DOOR);
  
  Serial.begin(9600);

  irrecv.enableIRIn(); 
  
  blenderservo.write(0);
  doorservo.write(0);
}

void moveforward() {
  digitalWrite(LFIN1, HIGH);
  digitalWrite(LFIN2, LOW);
  digitalWrite(LBIN1, HIGH);
  digitalWrite(LBIN2, LOW);
  
  digitalWrite(RFIN1, HIGH);
  digitalWrite(RFIN2, LOW);
  digitalWrite(RBIN1, HIGH);
  digitalWrite(RBIN2, LOW);
}

void movebackward() {
  digitalWrite(LFIN1, LOW);
  digitalWrite(LFIN2, HIGH);
  digitalWrite(LBIN1, LOW);
  digitalWrite(LBIN2, HIGH);
  
  digitalWrite(RFIN1, LOW);
  digitalWrite(RFIN2, HIGH);
  digitalWrite(RBIN1, LOW);
  digitalWrite(RBIN2, HIGH);
}

void turnleft() {
  //left wheels backwards
  digitalWrite(LFIN1, LOW);
  digitalWrite(LFIN2, HIGH);
  digitalWrite(LBIN1, LOW);
  digitalWrite(LBIN2, HIGH);

  //right wheels forwards
  digitalWrite(RFIN1, HIGH);
  digitalWrite(RFIN2, LOW);
  digitalWrite(RBIN1, HIGH);
  digitalWrite(RBIN2, LOW);
}

void turnright() {
  //left wheels forwards
  digitalWrite(LFIN1, HIGH);
  digitalWrite(LFIN2, LOW);
  digitalWrite(LBIN1, HIGH);
  digitalWrite(LBIN2, LOW);

  //right wheels backwards
  digitalWrite(RFIN1, LOW);
  digitalWrite(RFIN2, HIGH);
  digitalWrite(RBIN1, LOW);
  digitalWrite(RBIN2, HIGH);
}

void blend() {
  digitalWrite(BLENDIN1, HIGH);
  digitalWrite(BLENDIN2, LOW);
  Serial.println("blending");
}


// volume and skip buttons move robot front, back, left, right
// 0 button dispenses moss
// 1 blends
// on/off button

void remoteaction() {
  unsigned int value = results.value;
  Serial.println(value);
  
  switch (value) {
    case 32895: {
      Serial.println("moving forward");
      moveforward();
      break;
    }
      
    case 36975: {
      Serial.println("moving backward");
      movebackward();
      break;
    }
      
    case 8415: {
      Serial.println("turning left");
      turnleft();
      break;
    }
      
    case 24735: {
      Serial.println("turning right");
      turnright();
      break;
    }
      
    case 12495: {
      Serial.println("dispensing moss");

      if (!dispensing) {
        doorservo.write(90);
        dispensing = true;
        break;
      }
      else {
        doorservo.write(0);
        dispensing = false;
        break;
      }
    }
    
    case 16580863: {
      Serial.println("on/off");

      if (on) on = false;
      else on = true;
      break;
    }
      
    case 2295: {
      Serial.println("start blending");
      blend();
      blendtimer = millis();

      break;
    }
    
      
  }
}

void loop() {
  // blend for 20 seconds (simulation different)
  if (millis() - blendtimer <= 20000) {
    blend();
  }
  // open blender to tank for 20 seconds then close (simulation different)
  if (millis() - blendtimer > 20000 && millis() - blendtimer < 40000) {
    blenderservo.write(90);
    digitalWrite(BLENDIN1, LOW);
    digitalWrite(BLENDIN2, LOW);
  }
  else {
    blenderservo.write(0);
    digitalWrite(BLENDIN1, LOW);
    digitalWrite(BLENDIN2, LOW);
  }

  // get remote button presses
  if (irrecv.decode(&results)) {
    Serial.println(results.value);
    
    remoteaction(); 
    if (!on) {
      Serial.println("off");
      digitalWrite(LFIN1, LOW);
      digitalWrite(LFIN2, LOW);
      digitalWrite(LBIN1, LOW);
      digitalWrite(LBIN2, LOW);
      digitalWrite(RFIN1, LOW);
      digitalWrite(RFIN2, LOW);
      digitalWrite(RBIN1, LOW);
      digitalWrite(RBIN2, LOW);
      digitalWrite(BLENDIN1, LOW);
      digitalWrite(BLENDIN2, LOW);
      blenderservo.write(0);
      doorservo.write(0);
      
    }
    
    irrecv.resume(); // Receive the next value
  }
  
}

/*
 * Project: Remote for Beth's Moss Dispenser
 * 
 * Description: Remote for moss dispenser
 * Imagine the Tinkercad Circuit was built using an Arduino Nano
 * 
 * https://www.tinkercad.com/things/4zUE3ML0yFY
 * 
 * Pin mapping:
 * 
 * pin   | mode       | description
 * ------|--------    |------------
 * 5      input pullup   move forward   
 * 2      input pullup   turn left 
 * 8      input pullup   turn right
 * 4      input pullup   move backward
 * 6      input pullup   blender
 * 7      input pullup   dispense door
 * 9      input pullup   on/off
 * 
 * 
 * Credit:
 * /https://www.arduino.cc/en/Tutorial/Debounce
 * https://beatyourbit.wordpress.com/2018/02/05/arduinoinfrared/
 * https://www.instructables.com/id/IR-Transmitter-and-Receiver-Using-Arduino/
 */

#include <IRremote.h>

IRsend irsend;

const int FRONT = 5;
const int LEFT = 2;
const int RIGHT = 8;
const int BACK = 4;
const int DOOR = 6;
const int BLENDER = 7;
const int ON = 9;


unsigned long lastFrontTime = 0;
unsigned long lastBackTime = 0;
unsigned long lastLeftTime = 0;
unsigned long lastRightTime = 0;
unsigned long lastBlenderTime = 0;
unsigned long lastDoorTime = 0;
unsigned long lastOnTime = 0;

unsigned long debounceDelay = 50;

int lastFrontState = HIGH; 
int lastBackState = HIGH; 
int lastLeftState = HIGH; 
int lastRightState = HIGH; 
int lastBlenderState = HIGH;
int lastDoorState = HIGH; 
int lastOnState = HIGH;

int frontState;
int backState;
int leftState;
int rightState;
int blenderState;
int doorState;
int onState;



void setup()
{
  pinMode(LEFT, INPUT_PULLUP);
  pinMode(RIGHT, INPUT_PULLUP);
  pinMode(BACK, INPUT_PULLUP);
  pinMode(FRONT, INPUT_PULLUP);
  pinMode(DOOR, INPUT_PULLUP);
  pinMode(BLENDER, INPUT_PULLUP);
  pinMode(ON, INPUT_PULLUP);
  
  Serial.begin(9600);
  
}

void loop()
{
  bool frontReading = digitalRead(FRONT);
  bool backReading = digitalRead(BACK);
  bool leftReading = digitalRead(LEFT);
  bool rightReading = digitalRead(RIGHT);
  bool doorReading = digitalRead(DOOR);
  bool blenderReading = digitalRead(BLENDER);
  bool onReading = digitalRead(ON);
  

  if (frontReading != lastFrontState) {
    lastFrontTime = millis();
  }
  if (backReading != lastBackState) {
    lastBackTime = millis();
  }
  if (leftReading != lastLeftState) {
    lastLeftTime = millis();
  }
  if (rightReading != lastRightState) {
    lastRightTime = millis();
  }
  if (blenderReading != lastBlenderState) {
    lastBlenderTime = millis();
  }
  if (doorReading != lastDoorState) {
    lastDoorTime = millis();
  }
  if (onReading != lastOnState) {
    lastOnTime = millis();
  }
  
  //checking for new press of forward button
  if ((millis() - lastFrontTime) > debounceDelay) {

    if (frontReading != frontState) {
      frontState = frontReading;
      
      if (frontState == LOW) {
        Serial.println("moving forward");
        irsend.sendNEC(0x0000807F, 32);
      }
    }
  }
  
  if ((millis() - lastBackTime) > debounceDelay) {

    if (backReading != backState) {
      backState = backReading;
      
      if (backState == LOW) {
        Serial.println("moving backward");
        irsend.sendNEC(0x0000906F, 32);
      }
    }
  }
  
  
  if ((millis() - lastLeftTime) > debounceDelay) {

    if (leftReading != leftState) {
      leftState = leftReading;
      
      if (leftState == LOW) {
        Serial.println("turning left");
        irsend.sendNEC(0x000020DF, 32);
      }
    }
  }
  
  if ((millis() - lastRightTime) > debounceDelay) {

    if (rightReading != rightState) {
      rightState = rightReading;
      
      if (rightState == LOW) {
        Serial.println("turning right");
        irsend.sendNEC(0x0000609F, 32);
      }
    }
  }
  
  
  if ((millis() - lastBlenderTime) > debounceDelay) {

    if (blenderReading != blenderState) {
      blenderState = blenderReading;
      
      if (blenderState == LOW) {
        Serial.println("blending");
        irsend.sendNEC(0x000008F7, 32);
      }
    }
  }
  
  if ((millis() - lastDoorTime) > debounceDelay) {

    if (doorReading != doorState) {
      doorState = doorReading;
      
      if (doorState == LOW) {
        Serial.println("dispense button");
        irsend.sendNEC(0x000030CF, 32);
      }
    }
  }
  
  if ((millis() - lastOnTime) > debounceDelay) {

    if (onReading != onState) {
      onState = onReading;
      
      if (onState == LOW) {
        Serial.println("turn on/off");
        irsend.sendNEC(0x00000000, 32);
      }
    }
  }
  
  lastFrontState = frontReading;
  lastBackState = backReading;
  lastLeftState = leftReading;
  lastRightState = rightReading;
  lastBlenderState = blenderReading;
  lastDoorState = doorReading;
  lastOnState = onReading;
}

For the final project of Introduction to Physical Computing, we were tasked with building an assistive device for an elderly OSHER student. Within the scope of Arduino programing and design, we were to solve a personal problem for our client through custom designs. We were introduced to our client Diane, a former teacher who works as a docent at the Pittsburgh Museum of Natural History. Through interviews with Diane and discussion among the team, we landed on the ideation and creation of a wearable metronome bracelet for Diane to help her combat her tendencies to speed up or rush.

However, due to the major world event that is the COVID-19 pandemic, the final project had to be virtualize. Instead of creating a final tangible product, we will be creating the circuitry through simulations, and CAD modelings. This limitation leads to some uncertainties in our design that can be seen throughout the documentation.

The Final Metronome Bracelet

Our team made a metronome bracelet, which vibrates based on a beat that that Diane sets. It helps Diane from restlessness and lets her  focus better on what she is doing. She enjoys doing pilates, so we intend to make this in silicone, so that it is waterproof. We worked very hard to fulfill the clean, “cool” design aspect of this device to avoid the medical-device-esque design as Diane wanted.

The overall view of the product

The front view of the final product

The dimensions of the model

See-through view for the parts

The wiring of the parts

Tinkercad interactions

important to note the presence of the breadboard is not a part of the final design. It was implemented to make clear the connections.

Once the power is turned one, the user can control the frequency of the vibration. Note that the design requires us to us a Softpot Membrane instead of a normal potentiometer. However, due to Tinkercad’s simulation limitation, this is the closest alternative. The Softpot Membrane can be seen here:

https://www.sparkfun.com/products/retired/8607

PHYSICAL MOCKUP

Note that this was made to test the behaves-like prototype in person. As such, the prototype is much rougher than the final rendering.

Overall look of the bracelet. It has 5 small extruded braille dots (simulated through buttons on fabric). These buttons can change the frequency of beats, depending on where Diane touches them.

How Diane should interact with the bracelet. Note that the braille dots are much bigger than the actual design. These were the smallest button that I had access to in the house

Slide switch for Diane to turn on and off the bracelet. Located at the end of the button range to indicate orientation of the bracelet.

“Clasp” for Diane to adjust the length of the bracelet. Note that this is a button only for physical prototyping. This would ideally be a clasp, as indicated in the final rendering.

THE NARRATIVE SKETCH

Diane is giving a crowd a tour inside the museum. As she explains the rhythms and strokes of a painting, she notices herself speeding up. But she is not sure if that is just her thought, or if she is actually talking fast. She gives a single touch under her sleeve. The bracelet quietly gives her the beats in her personalized tempo that she can pace herself to. She finishes the presentation.

Diane goes to her pilates class. The instructor tells Diane that this position requires certain pace of breathing. The instructor says “inhale on one, two, three,” yet her breathing seems to be tangled at a certain beat. Diane touches her bracelet; this time, different from the museum tour, she adjusts the beat to a slower tempo. The instructor has no idea what kind of magic this bracelet does. She does not even think that it is a medical device, because it looks so cool! Diane finishes her class successfully.

The Process

From our last prototype, our project took a big shift after talking with Diane about what she would like to see in our design. While Diane was pleased with the aesthetics of the bracelet (She emphasized that this was very important to her), she didn’t find the bracelet’s custom frequency feature to be very useful. Rather, she was more interested in having set frequencies to shift through. She preferred having around 5 modes, ranging from fast to slow. 

With this new constraint, we researched new ways of interactive inputs beyond the simple button that we had previously worked with. After some digging, Ivan was able to find a part called a softpot potentiometer that worked well with our concept. This new part is a flexible potentiometer that takes in pressure inputs across a band. We found this to be perfect for Diane to wrap around her wrist and press along the band to get different frequency outputs.

Softpot potentiometer that Ivan found to be fitting for our project. It works similarly to a real potentiometer, where you can take in a range of inputs along the band.

With this, we broke down our tasks into three parts similar to the last phase. Elizabeth took the behavioral prototype, Chloe took the looks-like, and Ivan took the works-like. Since Elizabeth and Chloe’s work were interdependent on each other, Elizabeth’s work was frontloaded in the first week to hand off to Chloe in the next week. Our main goal was to be able to put all our parts together and receive feedback from Diane before presenting on the final day, so that we can adjust any minor issues prior to the presentation. 

Our GANTT chart. Ivan – works-like. Chloe – looks-like. Elizabeth- behaves-like.

We were able to maintain the schedule and workflow quite well, despite our timeline being pushed back by a couple days. When we checked in with Diane, however, she asked for no revisions, as she was happy with how the prototype turned out, putting us back on schedule.

Behaves-like prototype

The behavioral prototype began with considering how Diane would most intuitively interact with the bracelet. One obvious, yet easy-to-miss finding was that: Even though the flex potentiometer can wrap all the way around Diane’s wrist, the top of the wrist (where the watch face would usually sit) was the only real estate that allowed interactions. The fact that the potentiometer can be the bracelet in itself was deceiving me to think that Diane can press any point in the entire bracelet. This finding was not realized until after wrapping some papers around my wrist to feel how the bracelet might feel. 

top of wrist is the only interface I can work with, contrary to prior belief that the whole bracelet can be a potentiometer

During this phase, I was flexible with exploring how to include different outputs other than the regular metronome that we had planned. One suggestion from Diane from our discussion was that she would like to see variations of beats to support her inhales and exhales during pilates. As such, I considered having two potentiometers stacked on top of each other, where the top is the regular metronome and the bottom is the irregular breathing version. After talking with my dad, he suggested a much simpler version, where you include a switch to change the two modes. 

Sketch consideration for ergonomic layouts of buttons, as well as possibility of including a “breathing” version apart from the regular metronome

While this was thought about, Ivan brought up a good point that this irregular beats feature wouldn’t be very useful in different contexts of Diane’s life. As such, we simplified our features to its most essential part, so that it is clear, easy to use and lasting over time. 

The first prototype came through a physical prototype using fabric and buttons. Main lessons from sketching the bracelet was that simplicity was crucial for a small interface. As such, I kept the bracelet quite simple with 5 buttons that would lay on top of the wrist. 

My mom testing out the bracelet. She found it very straightforward and easy to use

From testing the bracelet with my parents, and it revealed two following points:

• The band is simple and easy to understand.
• My mom wanted the metronome beat to be fast inside the wrist, and slow down away from their body, whereas my dad preferred the opposite. How do I indicate the orientation of the bracelet in an intuitive way?

With this feedback in mind, I proceeded to make the model in 3D modelling. While the last prototype dealt with clarity of interactions, I wanted the modelling to take one step further in becoming aesthetic, as Diane wants. As such, I looked into various parametric bracelets for inspiration. I found out that Rhino and Grasshopper were a particularly useful tool for this, so I decided to try them out.

Inspirations I found online of bracelets that take advantage of curve forms.

Inspirations I found online of bracelets that take advantage of curve forms.

[Source1:

https://www.grasshopper3d.com/profiles/blogs/the-ghelix-bangle-sketching-in-grasshopper-and-3d-prints?overrideMobileRedirect=1 

Source2:

https://nodeform.com/collections/bracelets/products/stainless-steel-bracelet-bangle-3d-printed-morphed-design-ready-to-ship-medium-size]

My initial attempt was on Grasshopper, to get a parametric control over the form of the bracelet. I was following this tutorial: https://www.youtube.com/watch?v=CDnTFb3BgVo 

I had some problems with the initial steps, where the “polar points” component was acting oddly. Rather than creating an arc along the desired angle, the program was creating split arcs in full rotation. I couldn’t figure out how to fix this, so I decided to drop grasshopper for the time being.

According to the tutorial, the resulting line should’ve been a line of connected dots along a curve, but my program instead created a series of disconnected lines along a circle.

After doing another round of searching, I found this tutorial to be the most helpful: https://www.youtube.com/watch?v=Q8Issn3Mpzs. It was simple enough to follow, but allowed variation of thickness along the bracelet, which was my primary goal.

Main problems came from my unfamiliarity with the software. For example, I spent a good thirty minutes trying to figure out why my line was not divided into points as described in the video. It turned out that my view settings were different from the tutorial, so I had to dig around on my own to see how to make the points appear. 

The final model. Big to small waves indicate that the frequency range goes from slow to fast. This addressed the orientation issue previously found in user testing.

While the process was quite slow as I was new to the program, I was happy with how the bracelet turned out. I liked the bulbs that formed in the waves, making it more tactile than the traditional bracelet we imagine. However, I didn’t give much attention to how electronics would be held inside, which was a critical issue that Chloe had to work around after I passed off the design to her. 

Looks-like prototype

As for rendering the final model, Chloe’s main goal was to create a simple and compact design. While Elizabeth’s 3D prototype in Rhino fit the aesthetic goals, it didn’t meet the practical goals to build off of. For one, it was difficult to maintain the wavy form all the while holding all the electronics inside. Also, the thinnest parts of the bracelet would be prone to ripping, if the bracelet were to be made of silicone as intended. As such, Chloe remodelled the bracelet to be aesthetic in a simpler form. Main constraints came from making precise housing for the exact dimensions of each electronic part. 

Chart of dimensions for each electronic part

General difficulty making the model with exact dimensions. Vibration motor is at the bottom of the wrist since the skin is the thinnest and therefore most sensitive.

To solve the issue of orientation, Chloe put a gray dot in the bracelet to indicate that this is the faster frequency where the range begins. Diane would be able to know this intuitively through extended use over time. 

Gray dot placed to indicate orientation

Works-like prototype

For building the technical part of the bracelet, Ivan’s main constraint came from the size of the bracelet, as normal electronics would be hard to fit inside. After talking with Zach, Ivan decided to use the attiny, rather than the regular Arduino uno (or even the Arduino mini), as it had the smallest form factor.

Motor doesn’t respond when the power is directly connected to the attiny. Required a transistor connected to the coin cell battery to solve the issue.

Using the attiny required some troubleshooting, however, as when it was directly connected to the vibration motor, the motor was not responding to the inputs. Suspecting that the attiny was unable to power the motor properly, Ivan tested out the attiny with an LED bulb that requires less power and found his speculation to be correct. In order to readjust from there on, he used a transistor to connect to the motor’s power from the cell battery, rather than connecting the motor’s power to the attiny.

Conclusion

The final crit gave us good insight into the strengths and weaknesses of our project. Most everyone that commented really enjoyed the simplicity and elegance of the design, and found it to be useful in a variety of contexts.

Comments from crit and our responses

“I like the sleek and compact design. However, how do you change the batteries out? Is there a door of some sort? Would it be possible to use rechargeable cells and implement wireless charging of some sort?”

Lots of concerns were raised about changing the batteries. In the final design, there would be an opening to change the batteries when necessary. I’m not sure if it would’ve been possible to make this device rechargeable, as I’m not sure what component would be used to make that. It brings up a good question of how the user knows when the thing is out of battery versus just malfunctioning. Maybe a battery indicator would’ve been a nice touch to include.

“Your device is interesting in the way that it allows you to customize frequencies. But considering that the origin of this idea was the fact that she focused on a song have you thought about including different grooves, like a swing beat, or a clave?”

This was an interesting suggestion. I think we could have gone in the direction where Diane gives us 5 theme songs that she typically has in her head and we translate them into beats… But this may require more processing power, making the bracelet less compact as we designed for. Moreover, having a simple metronome matched closer to her original intent of wanting to calm down through regular beats.

“I love the design of the first bracelet as well as the second, and Diane seems to really love this! I am still confused, is the vibration beat already programmed, and she is able to change the frequency?”

This comment was similar to one of the questions we received during the critique session, where one of the Osher students wasn’t sure how Diane would pick the range of frequency for the metronome. While we discussed with Diane that the beats shouldn’t be too fast, that was the extent of our conversation regarding the range. Perhaps there could have been another potentiometer that sets the minimum frequency…

“Nice! Diane obviously loves it–most important! I’m not a Fit Bit user, but assuming that a Fit Bit doesn’t have this functionality? Also could there be a music component, for when Diane would like to relax with other sounds and not just beats? I loved the first design with the different size dots. I also wonder if these could be good for musicians who need/use metronomes.”

From our research, FitBit doesn’t have the exact metronome functionality, but there is an existing metronome bracelet geared towards musicians. (https://www.youtube.com/watch?v=E_nLhiYdfuY) It has a very large interface and noticeable LED lights, mainly for DJs to know the beats when they are performing. Our project was meant to be surreptitious in Diane’s life, as Diane did not want something that catches someone’s attention. As for the other suggestion of having sounds, that would also go against Diane’s desire to make this device be invisible. (Even if this was somehow connected to her headphones, that would make this useless in her primary use case of working with other people.)

Overall reflection

As for the remote conditions of this project, we found ourselves having very distinct works, rather than cross collaborating as projects usually go. For instance, Elizabeth and Chloe worked on the design one after another, rather than with each other, as it was difficult to design a 3d model of something together remotely. This led to some significant design changes from one person to another, and while the design improved in the second phase, it would have been more fruitful if we had collaborated in-person. Moreover, at times when all of us wanted to discuss functionalities of the design, we ran into some frustrations where we had trouble understanding each other, as we didn’t have the usual whiteboard table to support our ideas.

As for working with Diane, we didn’t find the age gap to be too different from working with any other client, especially since Diane had a very non-age-specific problem that we had to address. (Being restless is something that is probably more common in younger people than older people, so we found it to be less about working with someone that is older, but more about someone who is full of energy.) Diane was very helpful in designing with us on the product, as she painted a very rich picture of what her life looks like, making it easy for us to make informed choices during the design phase. Diane likes having theme songs in her head. She works as a museum docent, but she can become pretty restless in public situations. She likes fashion and aesthetics, so she doesn’t want anything that screams a medical device… We connected these helpful dots early on to create the metronome, rather than some music playing device as typical “calming” products go, making our project unique but fitting for Diane’s lifestyle.

Overall, we think that we made the right choice in maintaining the simplicity of the bracelet. Throughout the process, there were some doubts within the group that the bracelet might be too simple for a final project, leading us to think of different features to add. However, all of those features were considered extraneous and taken out in the end, as a simple and clear design is much more useful than a complex and difficult one. This turned out to be a good decision, as every feedback we received mentioned how they appreciated the simplicity of the device. We hope to apply this learning in future projects – hopefully when remote conditions end, allowing us to actually see the project in real life.

Technical Details

Tinkercad

Again, it is important to note the presence of the breadboard is not a part of the final design. It was implemented to make clear the connections.

TinkerCad Breadboard Rendering

Link for Tinkercad: https://www.tinkercad.com/things/aW7kHITO1Dl

code

The code found in tinkercad can be found here:

//Wearable Metronome 
// 
//Takes an analoge input from the the pot membrane 
//and turns the input into the frequency the vibrating 
//motor would turn on. Note this code is for a ATTINY 85 
//processor 
// 
// * pin mapping: 
// * 
// * pin | mode   | description 
// * 3   | input  | potentiometer
// * 2   | output | npn transistor for vibrating motor 
//

void setup()
{
  pinMode(3, INPUT);
  pinMode(2, OUTPUT);
}

void loop()
{
  unsigned int freq = map(analogRead(3), 0, 1000, 500, 5000);
  analogWrite(2, 255);
  delay(500); // Wait for 1000 millisecond(s)
  analogWrite(2, 0);
  delay(freq); // Wait for 1000 millisecond(s)
}

However, since the tinkercad version uses a normal potentiometer, whereas our final product needs to use a softpot membrane, which doesn’t store the input, we need to change our code a little bit. We added a storage variable to keep track of the selected state of the softpot membrane.  This code hasn’t been tested due to the inability to get our hands on a softpot membrane due to the COVID-19 situation, so there could be errors with the mapping.

//Wearable Metronome
//
//Takes an analoge input from the the pot membrane
//and turns the input into the frequency the vibrating
//motor would turn on. Note this code is for a ATTINY 85
//processor
//
// * pin mapping:
// * 
// * pin  |  mode   |  description
// * 3    |  input  |  softpot membrane
// * 2    |  output |  npn transistor for vibrating motor
//

int store;
unsigned int freq;

void setup()
{
  pinMode(3, INPUT);
  pinMode(2, OUTPUT);
}

void loop()
{
  if (analogRead(3) > 5){
    store = analogRead(3);
    freq = map(store, 0, 1000, 500, 5000);
  }
  analogWrite(2, 255);
  delay(500); // Wait for 1000 millisecond(s)
  analogWrite(2, 0);
  delay(freq); // Wait for 1000 millisecond(s)
}

Circuit schematic

Circuit Diagram depicting final circuit with softpot membrane

Design files

For our final project in Introduction to Physical Computing, our group (Leah, Abel and Varsha) was given the opportunity to work with an Osher student over the course of 5 weeks. Through our interview (documented here) we met our Osher student: Jane. She is a  retired English teacher, who likes to feed birds in her backyard. While feeding the birds using her bird feeders in the backyard, Jane had a problem with pests who would often come to eat the bird food that had spilled from her feeder. She also had an issue with larger animals and birds eating too much bird food from the feeder.  So, we focused on helping her solve this problem by building a prototype (documented here) of a bird feeder that would catch the bird food that fell off the feeder and release food only to the kinds of birds she wanted. This documentation will go over the final version of this project.

What we built

Description

Our bird feeder (download zip of CAD here) uses the benefits of technology to help it solve Jane’s problems. It can be hung on a tree or post and has a 10 inch hollow cylinder to hold bird food. It uses weight sensors to determine which types of birds/animals sit on it and accordingly releases food if it is a bird that Jane wants eating from the bird feeder. Then, if bird food falls from the bird feeder it is caught by a dish handing off the bottom of the feeder. After the dish becomes heavier due to food accumulation, a LED at the top of the feeder will light up, signaling for Jane to empty out the dish.

Images of our project

Image of the assembled bird feeder

Body of the feeder

Lid of the bird feeder

Base of the feeder

Close up of the funnel dispenser mechanism

Bird food catcher

Tinkercad Interaction Piece

Physical Mockups

Below is the video of the dispenser opening and closing.

The video below shows how the lid unscrews from the feeder.

This video shows how the catcher is detachable from the base.

Below shows how the catcher moves in the wind (based off the prototype).

Mockup dish moving in the wind

Narrative Sketch

Jane comes home after dropping her daughter at work and goes to the backyard. She notices that the she doesn’t have to fill up her bird feeder as much as she did before she got this new bird feeder and sees less pests around the house. She enjoys her day outside.

Two days, later Jane does to the backyard again and notices that a light is glowing on top of the bird feeder, this means she has to empty out the dish at the bottom. She goes and grabs a cup and tilts the dish deeply to empty it out of food. She then puts that food back into the feeder and goes on with her day.

How we got here

There weren’t a lot of redesigns and edits on the technical side, but the physical design went through a lot of changes. Originally, the feeder was not designed with a funnel. Thus, seed would spill out whenever the dispenser mechanism opened. Below is what the feeder looked like before the funnel was added.

The feeder and dispenser without the funnel

To ensure the seed was slowly dispensed without making a mess I drew up the designs depicted below.

Drawing of how the funnel is supposed to work.

The feeder is designed such that the bottom of the feeder will not fill up past the funnel. The holes in the feeder are positioned such that they are just above where the seed will fill to. Thus, a bird can stick its head in and eat the seed when the dispenser opens. This ensures all the feed won’t spill out when it opens. Furthermore, the lid had to be redesigned. Since the prototype, it was planned that the lid will be attached via a hinge and latch.

The top of the feeder before I created the lid.

Upon finishing the feeder and about to start the lid, we changed the design to make it more sturdy. A screw on lid would be sturdier and less likely to detach while in use. The base of the feeder was also redesigned.

My original design of the base.

Originally, the base had a lot of detail. It looked nice, but it wasn’t necessary and it made it more difficult to attach the dish. Thus, the base was simplified in the final version.

The top of the feeder before I added the LED

The last edit made to the feeder was to add the LED. It was left to the end, so we added it on last minute. The original dish design is shown below.

We also changed the design of the bird food catcher because it would be difficult to empty out due to its thick shape.

First draft bird dish

It would also be more heavy than necessary for the bird house. The hooks will be difficult put in and take out if Jane ever wants to do that. Thus, it was redesigned to the final version seen in the final CAD model above.

Conclusions and Lessons Learned

“It’s hard to think about how this is better or worse than the many bird feeders on the market.  It really needs to be field tested.”

Since we did not have access to the lab or the relevant supplies, we were unable to test our project and determine how it compares to other feeders. We attempted to research and implement the best improvements we could think of. However, we sadly can’t say how this will face the real world.

I am concerned that the door sliding up and down might startle the birds and keep them from the feeder.

This could scare away the birds, but we believe that the motor speed can be set to a low enough value with PMW to avoid this.

I think the food mechanism is very smart and it is a very cute design. But I wonder if the bird standing area is too small, and whether the weight sensor will affect all the food slots and how do you separate them?

The weight sensor will affect all the food slots since food can be dispensed by any weight sensor.

I like the research that red is the best color. I think a common concern is making sure only birds are able to be on the platform.

Since the sensor provides an analog value, we planned on determining the threshold for a bird to prevent food from being dispensed for another animal.

• We did not face any difficulties from working remotely as group members. Each member was able to work together well online since the parts were split up. However, we did have challenges with completing this project remotely from our client. We did not get the opportunity at the beginning of the project to see her garden due to Zoom difficulties and were limited during our first interview, which was over the phone.
• One major takeaway was figuring out the questions to ask to find out problems we could build a solution for. This was a challenging aspect of the project since you have to understand the person’s routines. By understanding the client’s interests, we narrowed the scope to gardening. This experience showed us the important of understanding the routines and interests to help come up with a solution to a problem.
• Our project is physical and has to be used outside with animals. Due to the limitations of Tinkercad, we were unable to completely test the functionality of our project. We had many comments about this, so it may have been better to come up with a solution that could be easier to test. Tinkercad is better than no simulation, but it is slow and is missing many parts. If we were to do this again, we would definitely build a physical project because of the simulation limitations.

Technical Details

Tinkercad final schematic

Image of the breadboard

Circuit schematic

/*
 * Bird Feeder
 *
 * Tinkercad Circuit: https://www.tinkercad.com/things/fwrD4bi2l3y
 * 
 * Description: This code is the software we provide to power our final project bird feeder.
 * We substituted force sensitive resistors with potentiometers since Tinkercad does not have
 * force sensitive resistors. This code reads input from 4 potentiometers that represent whether
 * a bird is at one of the food openings. It also takes input from a 5th potentiometer that represents
 * whether the food plate is full. If a bird is detected and the plate is not full, food will be dispensed.
 * This code will spin the motor for one second. If the food plate is full, then a light will turn on to
 * indicate that it must be cleared before more food can be dispensed. The status of the sensors will be
 * checked every 500ms.
 * 
 * Pin mapping:
 * 
 * pin   | mode   | description
 * ------|--------|------------
 * A0     input     potentiometer/force resistor(bird) 1 
 * A1     input     potentiometer/force resistor(bird) 2 
 * A2     input     potentiometer/force resistor(bird) 3 
 * A3     input     potentiometer/force resistor(bird) 4 
 * A4     input     potentiometer/force resistor(food)
 * 3 .    output    food dispense motor
 * 12     output    LED output when food plate full
 * 
 */


const int MOTOR_PIN = 3;
const int POT1_PIN = A0;
const int POT2_PIN = A1;
const int POT3_PIN = A2;
const int POT4_PIN = A3;
const int POT5_PIN = A4;
const int LED_PIN = 12;

unsigned long timer = 0;
const int INTERVAL = 500;

const int MOTOR_SPIN = 1000;

unsigned long last_on = 0;


void setup()
{
  pinMode(MOTOR_PIN, OUTPUT);
  pinMode(POT1_PIN, INPUT);
  pinMode(POT2_PIN, INPUT);
  pinMode(POT3_PIN, INPUT);
  pinMode(POT4_PIN, INPUT);
  pinMode(POT5_PIN, INPUT);
  pinMode(LED_PIN, OUTPUT);
}

void loop()
{
  bool bird1 = analogRead(POT1_PIN) < 512;
  bool bird2 = analogRead(POT2_PIN) < 512;
  bool bird3 = analogRead(POT3_PIN) < 512;
  bool bird4 = analogRead(POT4_PIN) < 512;
  bool bird_detected = bird1 || bird2 || bird3 || bird4;
  
  bool plate_empty = analogRead(POT5_PIN) > 512;
  
  if (millis() >= timer){
    if(bird_detected && plate_empty) {
      if (millis() >= MOTOR_SPIN + last_on) {
        digitalWrite(MOTOR_PIN, HIGH);
        last_on = millis();
      } else {
        digitalWrite(MOTOR_PIN, LOW);
      }
    } else {
      digitalWrite(MOTOR_PIN, LOW);
    }
    
    if (!plate_empty) {
      digitalWrite(LED_PIN, HIGH);
    } else {
      digitalWrite(LED_PIN, LOW);
    }
    timer = millis() + INTERVAL;
  }
}

Our team’s design proposes a solution to regularizing Yale Cohen’s eating schedule. During our first discussion with Yale, we discovered that he has lived with an odd eating schedule for most of his adult life, and currently, he also has a problem with snacking.

Following our learning of Yale’s love of puzzles and etymology, we have created a device that powers on during mealtimes in Yale’s corrected eating schedule. If Yale puts the plate or bowl he ate his meal in on the device’s sensor during this time, it will reward him with a puzzle. Essentially, we are creating a device that uses a reward system to entice Yale to eat at the times he defines as acceptable.

Initial Meeting with Yale: https://courses.ideate.cmu.edu/60-223/s2020/work/initial-meeting-documentation-team-yale/

Prototype Documentation: https://courses.ideate.cmu.edu/60-223/s2020/work/team-yale-prototype-documentation/

2. What we built (5 pts.)

We built a sensor and a puzzle reward. The sensor senses whether a plate is present to check whether Yale has had his meal within the time he is supposed to and that he is adhering with good eating habits. The puzzle reward is a way to incentivize Yale to have better eating habits. When he completes his task of eating at the right time, the puzzle reward would allow Yale to choose from two different puzzles: a sequence puzzle or a hangman puzzle. For the sequence puzzle, Yale will be given three numbers and is supposed to figure out what is the last number in the sequence. For the hangman puzzle, Yale will be given the number of characters in the answer and he would have to figure out what the word is.

Physical Design

Final Design Sketch

Housing and Sensor Plate Front View

Right View

Housing and Sensor Plate Back View

Animated look at the Remote

Rendering of Device in Kitchen

Secondary Front View

Yale wakes up and heads into the kitchen. The puzzle box lights up letting him know its time to eat. He eats a bowl of cereal and places the bowl on the plate sensor. He proceeds to play a game of hangman.

Puzzle Reward

The video shows the game play of the sequence game:

The following video shows the gameplay for the hangman game:

Ultrasonic Sensor

The video shows how the circuit works: The ultrasonic sensor first senses the distance of the plates to the sensor. If the distance is less than 20cm, and the waiting time of the object is equal 10 secs, the reward for the day is displayed to Yale.

3. How we got here (5 pts.)

Puzzle Reward

Decision 1: the puzzles we wanted to include

During our initial meeting with Yale, one thing that stood out to us is how he really enjoys puzzles. We discussed the different puzzles that he liked and figured that he is both interested in words and in math. He is also very adventurous in terms of exploring new biking trails and open to any sort of puzzles.

When we were creating our initial prototypes, we had to decide which puzzle we wanted to prototype since there was a limited amount of time. Also, we want to be able to flesh out one puzzle well rather than have a bunch of half baked puzzle prototypes which would not give us good feedback from users who we are user testing on. At this point, the puzzles we were considering are:

1. Math puzzle
   • For this puzzle, we wanted to give Yale a math problem to solve since he has a strong mathematical background. However, we soon realise due to the lack of real estate on the LCD screen, it was very hard to effectively show a long math problem (such as the bridge crossing problem). Even though we are aware that there is a possibility of having the screen scroll to show more text, we think it would not be a wise decision to have too many moving digital components for someone like Yale. We then thought that having equations and sequences would be the few options that would fit into the limitation of screen width
2. Hangman
   • For this puzzle, Yale will be given a word and would have to guess what the word is. Just like hangman, he would have seven lives to guess the word. The reason we thought he might like this is mainly because he mentioned that he enjoys etymology, the study of the origin of words and the way in which their meanings have changed throughout history.
3. Mechanical puzzle
   • As we started brainstorming the different puzzles that Yale enjoys, we realise that a lot of these puzzles are just a google search away for him. This is why we wanted to think of something out of the box that is more physical. We came up with the idea of a mechanical puzzle where we will be using switches, potentiometers and tilt sensors. In order to win the game, Yale would have to place the components in the right configuration. Each time he submits his answers, instead of knowing which component is right, he will only be told how many components are correct
4. Fun facts
   • Instead of having all the rewards requiring Yale to spend time solving a problem, we wanted to provide an option where Yale is able to be rewarded without having to spend a lot of time with the system. For this option, we wanted to just display a fun fact.
5. Trivia questions
   • A middle ground between a puzzle that would take around five minutes and a fun fact thrown to Yale is questions that would take a minute or two to answer. This allows for some sort of thinking and interaction but yet would not take Yale a long time to answer. We thought of using some true or false questions or creating a series of multiple choice questions and have it be his reward.

In the end, we decided to prototype the mechanical puzzle, because it was something that we were not sure whether Yale would enjoy. After prototyping an engineering prototype and a physical prototype, we managed to talk to Yale to get his opinions. It was very vague as to whether he wanted a mechanical puzzle because he would say that it would be “interesting” because it is “something he has never seen”. This led us to a decision point whether to include the mechanical puzzle. The decision would affect the design of the device because it would require more components which might make the design more complicated.

We decided to take his vague opinions to be indicators that he was not very interested in exploring a mechanical puzzle and thus decided to work with things he was very specific about that he liked – words and math. We then pivot our rewards to include a sequence puzzle and the hangman game.

brainstorming LCD flow

Decision: components to use

After our decision to use the sequence puzzle and the hangman game as our reward, we had to look into what are the components that we want to use. The main goals of this decision are:

1. Make sure that we are able to reuse as many components across the two puzzles
2. Make sure that the components are very clear to the user
3. Make sure the flow of events is intuitive

Taking into consideration our goals, we came up with a series of options for the components we can use:

• Option 1: 1 or 4 knobs, one button → use encoder and pushbutton
  • One knob to go through numbers (0-9)
  • Three knobs to go through alphabets? (A-Z) Each knob will control around 8 alphabets
  • Button to move to the next space? And to submit
• Option 2: 2 buttons
  • One button to move to the next space and submit
  • One button to circle through options (A → Z → A)
• Option 3: 3 buttons
  • One button to move to the next space and submit
  • One button to go to the next option (A → B)
  • One button to go to the previous option (B → A)

In an ideal world where we would have access to multiple parts and are not constrained by Tinkercad components, it would be best for us to use a rotary encoder, and modify the first option that we have.

Therefore, for our components, we decided to use:

1. One knob to loop through numbers, from 0 to 9
2. One knob to loop through alphabets, from A to Z
3. One button to confirm the selection
4. One button to move to the next option, such as moving to next place in a digit or moving from the sequence puzzle option to the hangman puzzle option
5. One button to move to the previous option

components used

Sensor

Firstly, the ultrasonic sensor circuit was built with the physical Arduino components. Feed backs from the prototype helped to identify modifications like adding a small time delay (say 10 seconds) before showing the game. Another modification was to improvise the LCD display from 16×4 size that we  initially planned to use, to 16×2 available on tinkercad.

Physical coupling of the circuit

Schematic

Sketch

Iteraction and Appearance

Early design sketch with first iteration of remote

Early design sketch with Bluetooth Communication between Sensor plate and device.

The method of interaction with the design was the first problem that I had to tackle. The first ideas I had involved placing our components on the facade of the device; however, I quickly abandoned that idea because placing the buttons and knobs on the device would limit the flexibility with which Yale could interact with the device. Additionally, I was worried that being so close to the device would make it uncomfortable to use and read the LCD. My next idea was to create a remote control device. My first iteration was inspired by the design for the Nintendo GameCube controller. The reason that I used this video game controller as a point of reference is that video game controllers are generally designed ergonomically to encourage comfortable play. However, following discussions with Yale, I abandoned this iteration because he prefered to use a single-handed device. The final design came after looking at and studying some of the television remotes in my house, which were rectilinear and encouraged single-handed use.

The appearance of the device was an easier problem to solve than interaction; however, there were many iterations before the final design. The main purpose of the housing of the device initially was to host the LCD screen and the components. The purpose of the sensor plate was to sense whether or not Yale had eaten and activate the puzzles. The sensor plate accomplishes this by detecting whether or not Yale has placed a bowl or plate on it during the time in which the device is activated. As such, early on I made the decision to make it round and flat to visually complement the objects it detects. However, the housing unit changed dramatically. The first iteration was a simple box, and while it was functional and would work, it was too generic to be truly beneficial. I then tried a more organic, rounded shape to create a visual contrast between the device and the other devices in the kitchen;  however, the white on white aesthetic of the device drew the eye as it is, and the rounded shape might prove difficult to interact with. Once I abandoned the idea of placing the interactive components of the device on the housing, I was free to downsize the device and I experimented with a trapezoidal shape that allowed for the LCD screen to be read easier. This is the design for the housing I ultimately went with. At this time, I also made the decision to separate the housing from the sensor plate and connect them with a wire to encourage flexibility.

4. Conclusions and lessons learned (5 pts.)

Response to Feedback

The rendering looked great. That said, consider using a device that is internet-enabled, such as a Particle Photon, so that you can pull new puzzles down to keep the novelty from wearing off.

This is something that we really wanted to do but was unable to due to the limitations we faced when working with tinkercad. In order to combat that problem, we tried to use as much randomness as we can so that the sequence is not very predictable. Also, this would mean that we would not have to hardcode a bunch of sequences, which would take a lot of memory on the arduino. By being able to get puzzles online, we will also not be limiting the sequence puzzles to be merely based on a function. One interesting puzzle we can have is the sequence 3, 3, 5, 4, 4, _. This sequence shows the number of letters is the word one, two, three, four, five and so on, which means the answer would be 3. Being able to obtain interesting puzzles like these would be awesome, but we also have to account for the explanation for the answer of the puzzles, which would improve the usability of the puzzle.

The design is very simple and pleasing. It seems like the project will need you to program lots and lots of puzzles so it doesn’t get boring, so that might be difficult

It is true that we would need a wide variety of puzzles so that it would not get boring. I would say that the sequence puzzle might be easy to crack once Yale figures out what the equation is. Ways we have tried to make it more challenging is by limiting it to three lives and also having a time limit. Other ideas we have to make the sequence more challenging would be to have less of the sequence given, which may or may not be solvable. In terms of the hangman game, one way to make it more interesting is to incorporate different languages that still uses the alphabets.

I like how personal this intervention is to Yale’s lifestyle. It’s interesting that solving puzzles serves as a reward. I wonder whether the current input buttons are sustainable over time, as they can become cumbersome to engage with if you were to dial through every letter and number. 

It would be interesting to be able to integrate with an actual keyboard which would make typing a word so much easier. However, one thing we wanted to do is to keep the design minimal. Also, we wanted to make sure it will be a feasible project to work on using tinkercad. It would be nice to be able to possibly connect to Yale’s phone or laptop, which would make the system simple but given our expertise, this is not something feasible for us.

I love that  there are two parts connected by IR receivers and transmitters. Does that currently work in the code or is that limited by tinkercad?

Unfortunately, we were not able to have the IR receivers and transmitters working on tinkercad. We would like to get it working if we are able to work on it further because it would greatly improve the usability of the system and not have a bunch of wires running around.

The design is really nice. Are the times for the eating schedule set? I think it would be convenient if it was adjustable.

Currently, the times are not scheduled due to the restrictions we have on tinkercad. In a perfect world, we would be using the real time clock component to figure out the time of the day. It would be nice to also start customising when meal times are.

Having an endless supply of puzzles will keep it interesting and motivating.  Really like idea of connecting to the internet for more puzzle choices. The device helps with regulating meal time but does it help with the snacking problem?

Unfortunately, the snacking problem cannot be enforced but we hope that by regulating meal times, Yale would have less need to have to snack at off hours.

Working With an Older Person

Working with an older person was a unique and valuable learning experience. Something that surprised me about Yale in particular was his vast knowledge of code and some of our components due to his work in software development before he retired. This knowledge allowed him to be more involved in some of the more technical aspects of the project than I originally expected. Working with an older person with a lot of life experience was both helpful and challenging in the context of this project in particular. The Puzzle Box is an assistive device that is attempting to modify the habits of our client Yale. Designing a device that can successfully modify somebody’s habits over time is a challenging problem to tackle to begin with; however, this challenge becomes much harder when one is trying to change habits that have been set for years or even decades. However, the long life that an older person has lived means that they are very familiar and sure of their interests, their likes, and their dislikes. This came especially helpful in our design of the reward system. Yale has established his love of puzzles for many years, and while he was open to a variety of puzzles and was willing to try new things, he also knew exactly what kind of puzzles he likes and that have kept his interest for so long.  Next time we work on designing an assistive device for an older individual, we should still take advantage of their knowledge of their interests in order to design a pleasing and appropriate device.

Working Remotely

Working remotely has been unexpectedly better than we had thought. Possibly because of the amazing team and a lively client that we had. Although, we cannot for sure say it was 100 percent because it had its challenges but one major advantage we had working remotely was that it was an opportunity to think outside the box and to put into practice what we learnt in the first half of the semester. Times when we had to improvise and make some necessary adjustments. 

Some major challenges we had were scheduling meeting times to reach the client, internet connectivity issues, easy understanding of client’s needs and limited resources/parts to work with. Due to the different calendar, we had to reschedule some meeting times with the client. Also, there was a time we had difficulty understanding what the client actually wanted which we thought could have been easier if we had a real physical conversation. Also, there was a time we had internet issues while talking to the client, and that also hindered the conversation. 

Another major challenge was limitation of parts and resources. Probably because the course as a whole is project based, working remotely did not help much with that. As good as tinkercad was, it had some limitations, for example, some components we could have used if we had physical access to the lab, were not present in the tinkercad. Also, one challenge we had was a storage problem. The original idea was to have different combinations of words as puzzles for Yale, but we realized that there would be a larger memory need for that word permutations. Yale suggested a cloud service option, but that would be out of scope for the objectives of this project.  

What worked better than we expected was the cooperation among the team and designation of the work. It was unexpectedly better because despite the distance and present situation of things, each team member delivered what was expected to make things work. We could also reach out to one another for help and suggestions. 

What we might have done differently in retrospect to allow remote collaboration work more successfully would be to have a team progress report or update. With that, it would be easy to track each team member’s progress. 

5. Technical details (4 pts.)

Ultrasonic Sensor code to detect plate.

// Ultrasonic sensor code to detect plates
// This code detects the presence of the plate. Usually there is a time lag of about 10secs before the game 
// is displayed to Yale (Client). 

#include <Wire.h>
#include <LiquidCrystal_I2C.h>

/*
   Create an LCD display object called "screen" with I2C address 0x27
   which is 16 columns wide and 2 rows tall.
*/

LiquidCrystal_I2C lcd(0x27, 16, 2);

// defining the Trig and Echo pins.
const int triggerPin = 8;
const int echoPin = 9;
const int buzzer = 10;
long duration = 0;
int distance = 0;

unsigned long previous_timer = 0;
unsigned long previous_timer_bf = 0;
unsigned long interval = 10;
unsigned long breakfastTime = 20; //assuming he eats his breakfast at 20secs (o'clock) everyday
unsigned long game_display_interval = 10;
unsigned long previous_timer_reset = 0;

// function to read the distance
long readUltrasonicDistance(int triggerPin, int echoPin)
{
  pinMode(triggerPin, OUTPUT);  // Clear the trigger
  digitalWrite(triggerPin, LOW);
  delayMicroseconds(2);
  // Sets the trigger pin to HIGH state for 10 microseconds
  digitalWrite(triggerPin, HIGH);
  delayMicroseconds(10);
  digitalWrite(triggerPin, LOW);
  pinMode(echoPin, INPUT);
  // Reads the echo pin, and returns the sound wave travel time in microseconds
  return pulseIn(echoPin, HIGH);
}


void setup() {
  // initializing the lcd:
  lcd.init();
  lcd.backlight();
  Serial.begin(9600);

  // set up the LCD's number of columns and rows:
  lcd.begin(16, 2);
  pinMode(buzzer, OUTPUT);
}

void loop() {

  previous_timer = millis() / 1000;
  
  // set the cursor to column 0, line 1
  // (note: line 1 is the second row, since counting begins with 0):
  lcd.clear();
  lcd.setCursor(0, 0);
  lcd.print("Hello Yale");
  

  // measure the ping time in distance
  distance = 0.01723 * readUltrasonicDistance(2, 3);

  if (millis() / 1000 == breakfastTime) {
    lcd.clear();
    lcd.setCursor(0, 1);
    lcd.print((String)"Time 4 Breakfast");
    tone(buzzer, 1000); // Send 1KHz sound signal...
    noTone(buzzer);     // Stop sound...
    while (millis() / 1000 < previous_timer + interval) {
      //wait approx. [period] ms
    }
  }

  if (distance <= 20 && (millis() / 1000) - previous_timer_bf >= game_display_interval) {
    lcd.clear();
    lcd.setCursor(0, 1);
    lcd.print((String)"Game 4 today is:");
    while (millis() / 1000 < previous_timer + interval) {
      //wait approx. [period] ms
    }
    previous_timer_bf = millis() / 1000;
  }

}

sensor schematics

Puzzle Reward System code

// include the library code:
#include <LiquidCrystal.h>

// initialize the library with the numbers of the interface pins
LiquidCrystal lcd(12, 11, 5, 4, 3, 2);
  
const int NUM_ROT = A0;
const int ALP_ROT = A1;

const int LEFT_BUT = 6;
const int RIGHT_BUT = 7;
const int CONFIRM_BUT = 8;
const int SUBMIT_BUT = 9;

// SUBMIT BUTTON THINGS

int buttonState;
int lastButtonState = LOW;
unsigned long lastDebounceTime = 0;
unsigned long debounceDelay = 20;

// RIGHT BUTTON THINGS

int rightButtonState;
int rightLastButtonState = LOW;
unsigned long rightLastDebounceTime = 0;
unsigned long rightDebounceDelay = 20;

// HANGMAN THINGS

String alphabets[] = { "A", "B", "C", "D", "E", "F", "G", "H", "I", "J", "K", "L", "M", "N", "O", "P", "Q", "R", "S", "T", "U", "V", "W", "X", "Y", "Z"};
String words[] = {"HELLO", "THERE", "SOMETHING", "SOMETHING", "NOTHING", "LOVELY", "POOPY", "NICE", "GIRL", "GIRLFRIEND"};
const int NUM_WORDS = 10;
String answer;
String guess;
String curr;

// GENERAL THINGS

int seqLife = 3;
int hangmanLife = 7;
bool gameOver = false;
bool gameWon = false;
String gameMode;
String gameModeChoices[] = {"HANGMAN", "SEQUENCE"};
int gameModeChoice = 0;
bool gameModeChosen;

unsigned long gameTime;
const int timeLimitSecs = 300;

byte heart[8] = {
  B00000,
  B00000,
  B01010,
  B11111,
  B11111,
  B01110,
  B00100,
  B00000
};

byte clockFace[] = {
  B00000,
  B01110,
  B10101,
  B10101,
  B10111,
  B10001,
  B01110,
  B00000
};


// SEQUENCE THINGS

int seqAnswer[4] = {1, 2, 3, 4};
int seqCurr;
int seqGuess = 0;
int seqCursor = 0;

void setup() {
  lcd.createChar(0, heart);
  lcd.createChar(1, clockFace);
  lcd.begin(16, 2);
  
  pinMode(LEFT_BUT, INPUT_PULLUP);
  pinMode(RIGHT_BUT, INPUT_PULLUP);
  pinMode(CONFIRM_BUT, INPUT_PULLUP);
  pinMode(SUBMIT_BUT, INPUT_PULLUP);

  pinMode(NUM_ROT, INPUT);
  pinMode(ALP_ROT, INPUT);

  Serial.begin(9600);
  
  createHangmanAnswer();
  createSequenceAnswer();
}

void createHangmanAnswer() {
  answer = words[random(0, NUM_WORDS)];
  for (int i=0; i < answer.length(); i++) {
    guess += "_";
  }
  Serial.println(answer);
}

void createSequenceAnswer() {
  int operation1 = random(4);
  int operation2 = random(4);
  int number1 = random(1, 10);
  int number2 = random(1, 10);

  if (operation1 == 0) {
    for (int i=0; i < 4; i ++) {
      seqAnswer[i] = seqAnswer[i] + number1;
    }
  } else if (operation1 == 1) {
    for (int i=0; i < 4; i ++) {
      seqAnswer[i] = pow(seqAnswer[i], number1);
    }
  } else if (operation1 == 2) {
    for (int i=0; i < 4; i ++) {
      seqAnswer[i] = seqAnswer[i] * number1;
    }
  } else if (operation1 == 3) {
    for (int i=0; i < 4; i ++) {
      seqAnswer[i] = seqAnswer[i] - number1;
    }
  }

  if (operation2 == 0) {
    for (int i=0; i < 4; i ++) {
      seqAnswer[i] = abs(seqAnswer[i] + number2);
    }
  } else if (operation2 == 1) {
    for (int i=0; i < 4; i ++) {
      seqAnswer[i] = abs(seqAnswer[i] - number2);
    }
  } else if (operation2 == 2) {
    for (int i=0; i < 4; i ++) {
      seqAnswer[i] = abs(seqAnswer[i] * number2);
    }
  } else if (operation2 == 3) {
    for (int i=0; i < 4; i ++) {
      seqAnswer[i] = abs(pow(seqAnswer[i], number2));
    }
  }

  Serial.println((String) operation1 + " " + operation2 + " " + number1 + " " + number2);
  Serial.println((String) "sequence!: " + seqAnswer[0] + " " + seqAnswer[1] + " " + seqAnswer[2] + " " + seqAnswer[3] + " ");
}

void loop() {
//  Serial.println((String) "Your word is: " + answer);

  if (gameModeChosen) {
    int alpReading = analogRead(ALP_ROT);
    int numReading = analogRead(NUM_ROT);
  
    if (gameOver) {
      gameOverDisplay();
    } else {
      if (gameMode == "HANGMAN") {
        playHangman(alpReading);
      } else if (gameMode == "SEQUENCE") {
        playSequence(numReading);
      }
    }
  } else {
    chooseGameMode();
  }
}

/* GENERAL THINGS */

void gameOverDisplay() {
  lcd.clear();
    lcd.setCursor(0,0);
  if (!gameWon) {
    lcd.print("GAME OVER, YALE");
  } else {
    lcd.print("CONGRATS, YALE!");
  }
}

void chooseGameMode() {
  lcd.setCursor(2,0);
  lcd.print("Hangman");
  lcd.setCursor(2, 1);
  lcd.print("Sequence");

  modeRightButtonPressed();
    
  lcd.setCursor(0,gameModeChoice);
  lcd.print("-");

  modeConfirmButtonPressed();
}

void modeRightButtonPressed() {
  int reading = digitalRead(RIGHT_BUT);
  if (reading != rightLastButtonState) {
    rightLastDebounceTime = millis();
  }

  if ((millis() - rightLastDebounceTime) > rightDebounceDelay) {
    if (reading != rightButtonState) {
      rightButtonState = reading;
      if (rightButtonState == LOW) {
        lcd.clear();
        gameTime = millis();
        if (gameModeChoice == 0) {
          gameModeChoice = 1;
        } else {
          gameModeChoice = 0;
        }
      }
    }
  }
  rightLastButtonState = reading;
}

void modeConfirmButtonPressed() {
  int reading = digitalRead(CONFIRM_BUT);
  if (reading != lastButtonState) {
    lastDebounceTime = millis();
  }

  if ((millis() - lastDebounceTime) > debounceDelay) {
    if (reading != buttonState) {
      buttonState = reading;
      if (buttonState == LOW) {
        gameMode = gameModeChoices[gameModeChoice];
        gameModeChosen = true;
        lcd.clear();
      }
    }
  }
  lastButtonState = reading;
}

/* SEQUENCE GAME THINGS */

void playSequence(int numReading) {
  
  seqCurr = numReading/103;
  lcd.noBlink();
  lcd.setCursor(0, 1);
  lcd.print(seqGuess);

  lcd.setCursor(seqCursor, 1);
  lcd.print(seqCurr);
  // Serial.println((String)"current!: " + seqCurr);

  lcd.setCursor(0, 0);
  lcd.print((String) seqAnswer[0] + "," + seqAnswer[1] + "," + seqAnswer[2] + ", __");
  

  sequenceCheckRightButtonPressed();
  sequenceCheckConfirmButtonPressed();

  lcd.setCursor(12, 0);
  lcd.write(byte(0));
  lcd.setCursor(14, 0);
  lcd.print(seqLife);

  lcd.setCursor(12, 1);
  lcd.write(byte(1));
  lcd.setCursor(13, 1);
  lcd.print(timeLimitSecs-(millis() - gameTime)/1000);
}

void sequenceCheckRightButtonPressed() {
  int reading = digitalRead(RIGHT_BUT);
  if (reading != rightLastButtonState) {
    rightLastDebounceTime = millis();
  }

  if ((millis() - rightLastDebounceTime) > rightDebounceDelay) {
    if (reading != rightButtonState) {
      rightButtonState = reading;
      if (rightButtonState == LOW) {
        Serial.println((String) "NEXT NEXT " + curr);
        seqGuess = seqCurr * 10;
        seqCursor += 1;
        Serial.println("seqGuess: " + String(seqGuess));
      }
    }
  }
  rightLastButtonState = reading;
}

void sequenceCheckConfirmButtonPressed() {
  int reading = digitalRead(CONFIRM_BUT);
  if (reading != lastButtonState) {
    lastDebounceTime = millis();
  }

  if ((millis() - lastDebounceTime) > debounceDelay) {
    if (reading != buttonState) {
      buttonState = reading;
      if (buttonState == LOW) {
        Serial.println((String) "curr is: " + seqCurr);
        Serial.println((String) "guess is: " + seqGuess);
        seqGuess += seqCurr;
        Serial.println((String) "new guess is: " + seqGuess);

        sequenceSubmitGuess();
      }
    }
  }
  lastButtonState = reading;
}

void sequenceSubmitGuess() {
  if (seqAnswer[3] == seqGuess) {
    Serial.println("RIGHT ANSWER");
    gameOver = true;
    gameWon = true;
  } else {
    Serial.println("WRONG ANSWER");
    lcd.clear();
    seqLife -= 1;
    seqGuess = 0;
    seqCursor = 0;
    if (seqLife == 0) {
      gameOver = true;
    }
  }
}

/* HANGMAN THINGS */

void playHangman(int alpReading) {
  curr = alphabets[alpReading/40];
  hangmanCheckButtonPressed();
  hangmanLcdDisplay();
  delay(10); 
}

void hangmanCheckButtonPressed() {
  int reading = digitalRead(CONFIRM_BUT);
  if (reading != lastButtonState) {
    lastDebounceTime = millis();
  }

  if ((millis() - lastDebounceTime) > debounceDelay) {
    if (reading != buttonState) {
      buttonState = reading;
      if (buttonState == LOW) {
        Serial.println((String) "curr is: " + curr);
        hangmanSubmitGuess();
      }
    }
  }
  lastButtonState = reading;
}

void hangmanSubmitGuess() {
  if (answer.indexOf(curr) != -1) {
    Serial.println("you got it!");
    Serial.println((String) "curr is: " + curr);
    guess = showCharInString(guess, answer, curr);
    Serial.println((String) "new guess is: " + guess);

    if (guess.indexOf("_") == -1) {
      gameOver = true;
      gameWon = true;
    }
  } else {
    Serial.println("does not exist");
    hangmanLife -= 1;
    if (hangmanLife == 0) {
      gameOver = true;
    }
  }
}

String showCharInString(String guess, String string, String chr) {
  String result = "";
  for (int i = 0; i < string.length(); i ++) {
    if (String(string[i])  == chr || String(guess[i]) != "_") {
      result += string[i];
    } else {
      result += "_";
    }
  }
  return result;
}

void hangmanLcdDisplay() {
  if (!gameOver) {
    lcd.setCursor(0, 0);
    lcd.print(guess);
    lcd.setCursor(0, 1);
    lcd.print(curr);

    lcd.setCursor(12, 0);
    lcd.write(byte(0));
    lcd.setCursor(14, 0);
    lcd.print(hangmanLife);

    lcd.setCursor(12, 1);
    lcd.write(byte(1));
    lcd.setCursor(13, 1);
    lcd.print(timeLimitSecs-(millis() - gameTime)/1000);
  }
}

Schematics

puzzle reward schematics

What we built:

Description

Our project is a foot arrow keyboard. There is a 14 in x 10 in platform with four arrow keys arranged in one row. The keys are styled to look and function like a gas pedal in a car. On the back edge of the platform is a hinge with a kickstand, which is adjustable to allow Lynne to set the keyboard at any desired height. There are two functions to the keyboard: a computer input mode and a Simon Says mode. For the computer input mode, Lynne can connect the keyboard to her laptop and browse the internet using the arrow keys, or she can play games that only require the four arrow keys (like Tetris or Snake). For the Simon Says mode, the keyboard is preprogrammed to randomly generate arrow key patterns for Lynne to repeat after. The difficulty of the Simon Says game starts out easy with a short pattern to repeat and gets more difficult as she continues to play.

Images:

CAD Images:

Overhead view of Foot Keyboard

Right side view of Foot Keyboard

Closer view of Power and Mode switches

Close up view of Left Arrow Key

Demonstration  of joint movement for Arrow Keys and Kickstand:

TinkerCad Demo:

Narrative Sketch

After a long day at work, Lynne arrives home and considers how to spend the rest of her afternoon. She has spent the majority of the day using her hands and fingers on her work computer and the pain from her arthritis is very uncomfortable. After greeting Mendy and her two cats, she decides to give her hands a rest and occupy herself by using the Foot Keyboard! She powers on the device and plays a couple of rounds of Simon Says, to see if she can beat her record of 6 rounds from the previous day. She enjoys the fast-paced, challenging nature of the game.

A few rounds (and high scores) later, Lynne plugs the Foot Keyboard into her computer and switches the mode to ‘CPU Input’ Mode to play some online keyboard games, like Snake and Pac-man. After she feels like she has played enough games for the afternoon, she opens up her favorite online book and begins reading, using the Foot Keyboard to flip pages. When bedtime eventually rolls around, the pain in her hands has subsided, she unplugs her Foot Keyboard and prepares for bed.

How we got here:

There were multiple steps that we took to get to the final project after the initial design and prototyping such as multiple rewiring in TinkerCad, debugging the Code to work as intended, as well as changing up the design to create a believable, functional device for Lynne.

The TinkerCad wiring and the internal mechanic were made before the 3D Cad Model, which led to some mishaps as the button location didn’t match up with the model shown. Initially the ON/OFF and Mode button was a push button, but with some discussion with the team, we switched to a slide switch. In addition, before the code was added to the TinkerCad, the wiring for the push button was wired to work with a pull up input. We kept the pull up wiring and we decided to change the code to work with a pull up input. Once the design of the 3D Cad model was finished, we went back and changed the wiring to match up the location as close as possible. The Up and Down button location was swapped, as well as the indicated LED, in addition to the ON/OFF and MODE switch location. The wiring for the battery was also changed after creating the schematic, as I noticed that the ON/OFF button was not connected to the battery.

The very first draft of the TinkerCad wiring: The ON/OFF and MODE is push-button, and the arrow button is in a different order than the finalized Foot Keyboard.

For one section of the code: the computer input, we discovered that type of Arduino matters. While the code was simple with one add on of a library called “Keyboard.h” and input called “Keyboard.press” the code wasn’t able to be read with Arduino Uno. With research it was due to Uno’s different USB communication from other Arduinos. With further research Ardunio Leonardo, which has a certain USB communication that can read Keyboard and mouse inputs when connected to the computer, is the best option to use if we were to make it physically. 

Arduino Uno Port: Computer Input Code not working/verifying with multiple code errors.

Arduino Leonardo Port: Verified Code after the port has been switched to Arduino Leonardo.

Another issue we ran into was integrating the code with the Tinkercad model. To represent variables with different “states”, such as the current Simon Says game mode or the direction of a button or LED, we decided to use enums in our code. This way, we could use a switch-case statement to cleanly execute different blocks of code depending on which state a variable was. We tried to use two enums (game_stage and dir) and this compiled mess-free in the Arduino IDE, but unfortunately caused mysterious bugs when pasting the code into the Tinkercad model. Unable to figure out the issue on our own, with the help of Zach, our professor, and Harshine, our TA, we were able to get the Tinkercad model working by replacing the second enum (dir) with the use of an integer (with values 0,1,2,3).

Tinkercad would throw compilation errors when attempting to use a second enum for direction.

Switched from an enum to using an integer to represent the four directions.

Conclusions and lessons learned:

We received a lot of helpful feedback and ideas from the final critique. One interesting idea which emerged from discussion was “Have you thought about adding a handle to the device?” We thought this was an excellent suggestion, since: 1) a handle wouldn’t be too hard to add to the design (a wide rectangular hole cut from base on the opposite side of the power/mode switches) and 2) it would make moving the keyboard (perhaps from the ground to the recliner) an easier endeavour. Other helpful suggestions appeared in the written feedback. One critiquer suggested mitigating typing fatigue by using “a nicer (mechanical) keyboard that isn’t as tiring to type on (since you don’t have to bottom the keys out)?” This suggestion ties into another provided during discussion, which was “There are large pre-made keys you can purchase which would allow you to avoid using foam.” Typing fatigue is definitely an important consideration, especially for a keyboard which works the typically unused foot and calf muscles. A feature to fine-tune during actual fabrication would be the thickness of the foam beneath the key-pedals to find a balance between sensitivity and fatigue. If this balance turns out to be difficult to obtain, looking into purchasing manufactured keys would definitely be a slightly more expensive, yet reasonable, suggestion to consider. A fourth suggestion, found in the written feedback, was to add a fifth “spacebar since a lot of games also require that button.” During the design phase, we played around with the idea of adding more keys but concluded that four was an optimal number, in terms of keeping the size fairly small and avoiding typing fatigue as much as possible.

Working remotely has been fairly strange as physical projects are more easily done in person. Overall, this experience was a positive one. Coordinating with each other online wasn’t too difficult. We found most difficulty in not being able to collaborate on unshareable files together, such as the Fusion 360 model and our Arduino code. Other than those challenges, we worked fairly cohesively, and there is not much we would do differently given the chance to work together again. Our client, Lynne, was extremely fun to work with, and she easily kept in contact with us as we worked on our device. We would’ve loved to meet with her in person and show her our work face to face, but chatting with her through Zoom has still been an enjoyable experience. 

Our experience of working with an older individual was pleasant, as Lynne was very helpful in each of the steps we took, and how as a whole we were able to keep our conversation light and fun. What was difficult initially was coming up with a personalized device for Lynne from our first meeting, as many of our initial designs could have been used by the public. What helped us get through the initial challenge was, as mentioned before, how Lynne was easy to keep in contact with as we worked on our device. As we narrowed down our idea, we were able to ask additional questions, such as what games she enjoyed and what difficulties she may have with our initial foot keyboard design. Each critique we got from Lynne, we were able to slowly make the keyboard more personalized for her, which was a big help as we finalized our project. Because Lynne was tech-savvy there were no problems while working remotely, but if we were to do this again, during our initial interview, we could have asked multiple follow up questions, so we do not assume things, for example, when we initially designed it to be used on the floor rather than a recliner.

Technical details:

Design Files:

Final Project Lynne Laser Cut DXFs

TinkerCad:

TinkerCad Link: https://www.tinkercad.com/things/7FbfX3RFi6Y

TinkerCad: Foot Keyboard complete wiring

Schematic:

Foot Keyboard Final Schematic

Code:

/*
   Project Title: Foot Keyboard - Team Lynne

   Names: Sue Lee, Hojung Kim, Achilles Lin

   Description: This code controls the two modes of the Foot Keyboard. The Simon Says mode randomly
   generates sequences and asks the user to repeat the sequences in increments of 3 moves. The code
   receives user input from the 4 momentary pushbuttons and delivers output through 4 LEDs and the 
   LCD screen. The Computer Input mode allows the user to use the 4 momentary pushbuttons to send
   arrow key controls to a USB connected computer. The code continually loops to detect changes in 
   the mode switch, allowing the user to switch between modes at any point in either mode.

   Pin mapping:

   pin   | mode   | description
   ------|--------|------------
   10       input    Left Key Momentary Pushbutton
   9        input    Right Key Momentary Pushbutton
   8        input    Up Key Momentary Pushbutton
   7        input    Down Key Momentary Pushbutton
   A0       output   Left Key LED
   A1       output   Right Key LED
   A2       output   Up Key LED
   A3       output   Down Key LED
   A4       output   Mode Switch


   Credit: Randomized numbers code from https://www.arduino.cc/reference/en/language/functions/random-numbers/random/

   License Notice:
   Copyright 2020 Sue Lee, Hojung Kim, Achilles Lin

   Permission is hereby granted, free of charge, to any person obtaining
   a copy of this software and associated documentation files (the "Software"),
   to deal in the Software without restriction, including without limitation
   the rights to use, copy, modify, merge, publish, distribute, sublicense,
   and/or sell copies of the Software, and to permit persons to whom the
   Software is furnished to do so, subject to the following conditions:

  The above copyright notice and this permission notice shall be included in
  all copies or substantial portions of the Software.

  THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
  IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
  FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
  THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
  LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
  OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
  SOFTWARE.

*/

#include <LiquidCrystal.h>
#include <Keyboard.h>

LiquidCrystal lcd(12, 11, 5, 4, 3, 2);

#define LBTN_PIN 10
#define RBTN_PIN 9
#define UBTN_PIN 8
#define DBTN_PIN 7

#define LLED_PIN A0
#define RLED_PIN A1
#define ULED_PIN A2
#define DLED_PIN A3

#define MODE_PIN A4

bool prev_mode;

// Stages
enum game_stage {
  OFF,
  MODE_DISPLAY,
  PRE_START,
  ROUND_DISPLAY,
  PATTERN_DISPLAY,
  PATTERN_INPUT,
  CORRECT_FEEDBACK,
  GAMEOVER_FEEDBACK
};

/* tried to use enum but would crash tinkerCad
enum dir {
  UP,
  DOWN,
  LEFT,
  RIGHT
//  {0,1,2,3} -> {UP,DOWN,LEFT,RIGHT} respectively
}; */

// must be multiple of 3
#define MAX_PTN_LEN 30

enum game_stage curr_stage = OFF;
enum game_stage prev_stage = OFF;

int round_num = 1;
int pattern[MAX_PTN_LEN] = { }; // initialize as zeros
int pattern_index = 0;

unsigned long stage_tmr = 0;

// instructions constants
#define MODE_DISPLAY_INTVL 2000
#define ROUND_DISPLAY_INTVL 2000
#define CORRECT_DISPLAY_INTVL 3000
#define GAMEOVER_DISPLAY_INTVL 3000

// in game constants
#define PTRN_DISPLAY_INTVL 2000
#define PTRN_DISPLAY_PADDING 500
#define PTRN_INPUT_INTVL 5000

/* HELPER FUNCTIONS */

// returns the amount of time that has elapsed since stage_tmr was last set
unsigned long elapsed() {
  return millis() - stage_tmr;
}

// reset the stage_tmr with millis()
void reset_timer() {
  stage_tmr = millis();
}

// returns true if the current game stage doesn't match the previous game stage
bool new_stage() {
  return curr_stage != prev_stage;
}

// returns true if the current keyboard mode doesn't match the previous keyboard mode
bool new_mode() {
  return digitalRead(MODE_PIN) != prev_mode;
}

// clears the LCD screen and prints str onto the LCD
void clearPrintLCD(String str) {
  //insert LCD print code here
  lcd.clear();
  lcd.print(str);
}

// return the negated button state the desired key where 
// {0,1,2,3} -> {UP,DOWN,LEFT,RIGHT} respectively
bool readBtn(int key) {
  switch (key) {
    case 0:
      return !digitalRead(UBTN_PIN);
    case 1:
      return !digitalRead(DBTN_PIN);
    case 2:
      return !digitalRead(LBTN_PIN);
    case 3:
      return !digitalRead(RBTN_PIN);
  }
  return false;
}

// sets the LED output value of the desired key's LED where
// {0,1,2,3} -> {UP,DOWN,LEFT,RIGHT} respectively
void setLED(int key, bool output) {
  switch (key) {
    case 0:
      digitalWrite(ULED_PIN, output);
      break;
    case 1:
      digitalWrite(DLED_PIN, output);
      break;
    case 2:
      digitalWrite(LLED_PIN, output);
      break;
    case 3:
      digitalWrite(RLED_PIN, output);
      break;
  }
}

// sets the LED output values of all the key's LEDs
void setAllLEDs(bool output) {
  for (int i = 0; i < 4; i++) {
    setLED(i, output);
  }
}

// initializes the simon says game round number and pattern sequence
void init_game() {
  // reset round_num
  round_num = 1;

  // randomize pattern sequence
  Serial.println("Generating sequence: ");
  Serial.print("<");
  for (int i = 0; i < MAX_PTN_LEN; i++) {
    pattern[i] = random(4);
    Serial.print(pattern[i]);
    if (i < MAX_PTN_LEN - 1) {
      Serial.print(",");
    }
  }
  Serial.println(">");
}

void setup() {
  // initialize inputs/outputs
  Serial.begin(9600);

  pinMode(UBTN_PIN, INPUT_PULLUP);
  pinMode(DBTN_PIN, INPUT_PULLUP);
  pinMode(LBTN_PIN, INPUT_PULLUP);
  pinMode(RBTN_PIN, INPUT_PULLUP);

  pinMode(ULED_PIN, OUTPUT);
  pinMode(DLED_PIN, OUTPUT);
  pinMode(LLED_PIN, OUTPUT);
  pinMode(RLED_PIN, OUTPUT);

  pinMode(MODE_PIN, INPUT);

  lcd.begin(16, 2);

  // if analog input pin 0 is unconnected, random analog
  // noise will cause the call to randomSeed() to generate
  // different seed numbers each time the sketch runs.
  // randomSeed() will then shuffle the random function.
  // Note: tinkerCad generates the same "random" sequence each time
  randomSeed(analogRead(0));

  // reads the keyboard mode and initliazes global mode variables
  prev_mode = digitalRead(MODE_PIN);
}

void loop() {
  // read keyboard mode
  bool mode_switch_state = digitalRead(MODE_PIN);

  if (new_mode()) {
    if (mode_switch_state) {
      // Just entered Simon Says game mode
      // set game stage to MODE_DISPLAY
      curr_stage = MODE_DISPLAY;
    } else {
      // Just entered Computer Input mode
      prev_stage = OFF;
      clearPrintLCD("CPU INPUT MODE");
      Serial.println("Computer Input Mode");
    }
  }

  bool ubtn_state;
  bool dbtn_state;
  bool lbtn_state;
  bool rbtn_state;

  // Check if keyboard is in Simon Says Game Mode
  if (mode_switch_state) {
    switch (curr_stage) {
      // MODE_DISPLAY: print "SIMON SAYS MODE" to LCD for MODE_DISPLAY_INTVL
      case MODE_DISPLAY:
        if (new_stage()) {
          // When entering MODE_DISPLAY stage, print "SIMON SAYS MODE" to LCD
          prev_stage = curr_stage;
          reset_timer();
          clearPrintLCD("SIMON SAYS MODE");

          Serial.println("MODE DISPLAY STAGE");
        } else if (elapsed() > MODE_DISPLAY_INTVL) {
          curr_stage = PRE_START;
        }
        break;
        
      // PRE_START: print "PRESS -> TO START" until right button press (light up right LED with right button press)
      case PRE_START:
        if (new_stage()) {
          // When entering PRE_START stage, print "PRESS -> TO START" to LCD
          prev_stage = curr_stage;
          reset_timer();
          clearPrintLCD("PRESS -> TO START");

          Serial.println("PRE-START STAGE");
        }
        
        // upon right key button press: light up right LED, intialize game
        rbtn_state = readBtn(3);
        if (rbtn_state) {
          setLED(3, HIGH);
          while (rbtn_state) {
            rbtn_state = readBtn(3);
          }
          setLED(3, LOW);
          curr_stage = ROUND_DISPLAY;
          init_game();
        }
        break;
        
      // ROUND_DISPLAY: print "ROUND <round_num>" for ROUND_DISPLAY_INTVL
      case ROUND_DISPLAY:
        if (new_stage()) {
          // When entering ROUND_DISPLAY stage, print "ROUND <round_num>" to LCD
          prev_stage = curr_stage;
          reset_timer();
          clearPrintLCD("ROUND ");
          lcd.print(round_num);

          Serial.println((String)"ROUND " + round_num);
        } else if (elapsed() > ROUND_DISPLAY_INTVL) {
          curr_stage = PATTERN_DISPLAY;
        }
        break;

      // PATTERN_DISPLAY: print "SIMON SAYS..." to LCD and display <3x round_num> moves with LEDS: PTRN_DISPLAY_INTVL millis LED ON, PTRN_DISPLAY_PADDING millis LED OFF
      case PATTERN_DISPLAY:
        if (new_stage()) {
          // When entering PATTERN_DISPLAY stage: print "SIMON SAYS..." to LCD, reset pattern_index
          prev_stage = curr_stage;
          reset_timer();
          clearPrintLCD("SIMON SAYS...");
          pattern_index = 1;

          Serial.println("PATTERN_DISPLAY MODE");
        }
        
        // light up each LED corresponding to sequence
        if (pattern_index > (round_num * 3)) {
          // once all moves have been displayed for current round, move to next stage
          curr_stage = PATTERN_INPUT;
        } else if (elapsed() < PTRN_DISPLAY_INTVL) {
          // turn on LED
          setLED(pattern[pattern_index - 1], HIGH);

          Serial.println((String)"Pattern index: " + pattern_index + ", Key: " + pattern[pattern_index - 1]);
          delay(PTRN_DISPLAY_INTVL);
        } else if (elapsed() > PTRN_DISPLAY_INTVL) {
          // turn off LED
          Serial.println("Turn LED OFF");
          setLED(pattern[pattern_index - 1], LOW);
          pattern_index++;
          delay(PTRN_DISPLAY_PADDING);
          reset_timer();
        }
        break;

      // PATTERN_INPUT: print "YOUR TURN!" to LCD and accept user input of pattern, light up LEDs with button presses
      case PATTERN_INPUT:
        if (new_stage()) {
          // When entering PATTERN_DISPLAY stage: print "SIMON SAYS..." to LCD, reset pattern_index
          prev_stage = curr_stage;
          reset_timer();
          clearPrintLCD("YOUR TURN");
          pattern_index = 1;
        }
        
        if (pattern_index > (round_num * 3)) {
          // once all moves have been inputted for current round, move to next stage
          curr_stage = CORRECT_FEEDBACK;
        } else if (elapsed() > PTRN_INPUT_INTVL) {
          // game over if time runs out
          curr_stage = GAMEOVER_FEEDBACK;
        } else {
          ubtn_state = readBtn(0);
          dbtn_state = readBtn(1);
          lbtn_state = readBtn(2);
          rbtn_state = readBtn(3);

          int curr_dir = pattern[pattern_index - 1];
          int input_dir = 0;

          // check if any button has been pressed
          if (ubtn_state || dbtn_state || lbtn_state || rbtn_state) {
            if (ubtn_state) {
              Serial.println("UP");
              input_dir = 0;
            } else if (dbtn_state) {
              Serial.println("DOWN");
              input_dir = 1;
            } else if (lbtn_state) {
              Serial.println("LEFT");
              input_dir = 2;
            } else if (rbtn_state) {
              Serial.println("RIGHT");
              input_dir = 3;
            }

            // turn on corresponding key LED until button unpressed
            setLED(input_dir, HIGH);
            int pressed_btn_state = readBtn(input_dir);
            while (pressed_btn_state) {
                pressed_btn_state = readBtn(input_dir);
            }
            // turn off LED
            setLED(input_dir, LOW);

            // check if correct key was pressed
            if (curr_dir == input_dir) {
              // correct key pressed
              pattern_index++;
              reset_timer();
            } else {
              // incorrect key pressed
              curr_stage = GAMEOVER_FEEDBACK;
            }
          }
        }
        break;

      // CORRECT_FEEDBACK: print "NICE JOB!" for CORRECT_DISPLAY_INTVL
      case CORRECT_FEEDBACK:
        // print "NICE JOB!" for CORRECT_DISPLAY_INTVL
        if (new_stage()) {
          prev_stage = curr_stage;
          reset_timer();
          clearPrintLCD("NICE JOB!");

          Serial.println("CORRECT FEEDBACK STAGE");
        } else if (elapsed() > CORRECT_DISPLAY_INTVL) {
          curr_stage = ROUND_DISPLAY;
          round_num++;
        }
        break;

      // GAMEOVER_FEEDBACK: print "GAME OVER :(" and flash all LEDs on for GAMEOVER_DISPLAY_INTVL
      case GAMEOVER_FEEDBACK:        
        if (new_stage()) {
          prev_stage = curr_stage;
          reset_timer();
          clearPrintLCD("GAME OVER :(");
          setAllLEDs(HIGH);

          Serial.println("GAMEOVER FEEDBACK STAGE");
        } else if (elapsed() > GAMEOVER_DISPLAY_INTVL) {
          // turn off all LEDs
          curr_stage = PRE_START;
          setAllLEDs(LOW);
        }
        break;

      // default handles OFF stage; curr_stage should never be OFF
      default:
        clearPrintLCD("ERROR");
        Serial.println("ERROR");

        delay(500);
    }
  } else {                  // computer input mode
    Keyboard.begin(); //begin keyboard

    //Up Button
    if (readBtn(0)) {
      Keyboard.press(KEY_UP_ARROW);
      Serial.println("KEY_UP_ARROW PRESSED");
    }
    else {
      Keyboard.releaseAll();
    }

    //Down Button
    if (readBtn(1)) {
      Keyboard.press(KEY_DOWN_ARROW);
      Serial.println("KEY_DOWN_ARROW PRESSED");
    }
    else {
      Keyboard.releaseAll();
    }

    //Left Button
    if (readBtn(2)) {
      Keyboard.press(KEY_LEFT_ARROW);
      Serial.println("KEY_LEFT_ARROW PRESSED");
    }
    else {
      Keyboard.releaseAll();
    }

    //Right Button
    if (readBtn(3)) {
      Keyboard.press(KEY_RIGHT_ARROW);
      Serial.println("KEY_RIGHT_ARROW PRESSED");
    }
    else {
      Keyboard.releaseAll();
    }

    delay(200);
    Keyboard.end(); //stops keybord
  }
  
  prev_mode = mode_switch_state;
  delay(50);
}

We created an assistive device for an older adult that would improve a certain aspect of their daily routine or bring more positivity to their life. We collaborated with our Osher client, Fredrick, and communicated with him throughout the process of the project to help us figure out the design for our intervention. The problem we decided to tackle was how we could make Fredrick’s daily exercise routine more interesting by creating a guide and friend that could not only carry the routine sequence but make the process more engaging.

What We Built

Our device is a hand-sized interactive device that will act as your exercise guide and companion that will shuffle your exercise routine and encourage you to finish each session. The device will say the exercise title through speakers and it will also display the exercise title on its screen in the front. You communicate to the device when you have completed that exercise by making a loud sound (this means that the environment you are working out in should not have too many loud noises or disturbances). The device will track your overall progress with the exercise routines and at the very end, it will congratulate you that you have accomplished all the exercises.

Final Product

Our Exercise Companion device

How the device lights up

Where to change the batteries

The rotary encoder allows you to switch to a different set of routines and shuffle routine

Simulation of a button press to shuffle an exercise routine in a category

Simulation of button rotation to go to next exercise category

Animation of how the device works overall

TinkerCad

The main interaction stimulated in TinkerCAD.

The celebration when the workout is finished

Physical Mockups

A prototype of how the device could work

On one of the screens portrays a digital face

Prototype of how the device could look

Interacting with the button press

To learn more about our prototypes, click here 

Narrative Sketch

To map out all the interactions Frederick might have with the exercise companion, we developed storyboards of different use scenarios.

The exercise companion is powered on with a long press of the button. A personalized welcome message is displayed on the LCD screen and is announced on the speaker. The eyes on the device respond by blinking.

The user can move on to the next exercise by pressing the button once. The name of the exercise is announced on the speaker and LED lights of the progress bar indicates the progress through the exercise session.

Alternatively, the user can also skip to the next exercise with an audio input by saying “next” out loud.

The exercise companion celebrates with an audio and a visual message when the exercise session is complete and the lights on the progress bar are all lit up.

The user can switch between different exercise routines by turning the rotary dial on the exercise companion.

How We Got Here

We initially interviewed Fredrick and brainstormed multiple ideas with him.  In the end, we decided to pursue the exercise companion because it was the most feasible, and Fredrick also agreed that it would be something fun and useful for him.

Our Zoom interview with Fredrick

A sample of our interview

To see our full interview story click here

The basic concept was to have a device that would shuffle his exercise routines. He would press the button to shuffle the exercise routine and the device would countdown each exercise for Fredrick to follow along. There would also be LED lights to track the overall progression of the routine.  But as we got into the specifics, we changed and developed our device to better solve the problem:

A collaborative drawing done by all team members over Zoom. This incorporated all of our individual prototype designs.

1. Adding voice feedback

The first time we revealed to Fredrick what we were making, we had a button that he would be able to press every time he completed a workout. However, he raised the concern that it would be very cumbersome to get up to press the button every time, therefore, this was when we decided to use sound to go to the next workout.

Initially, we had a button but changed it to a rotary encoder to better fit Fredrick’s needs. The voice recognition would be inside the device near the speaker holes.

2. Instead of two touch screen faces on opposite ends, we used a 16×2 LCD screen

In our first final prototype, we decided to have a cubic screen with a face that acts as the “personality” of the device and changed according to the progression and accomplishment of the exercises. Another screen was in the back to have the title of the routines, in case Fredrick did not hear the exercises being called out through the speakers. However, we realized this was a little unnecessary and would add bulk to our physical size, therefore, we simplified this concept by using the form of the 16×2 LCD screen as the mouth of the device. The eyes of the mouth would be cut out from the body of the device and placed  with acrylic on top to be able to see LED lights inside. The LED lights in the eyes would then flash to animate the face.

Initial rendering of the device with two cubic screens, one in front and one in the back

Final design of device with one side of the cube with a face made from acrylic, a 16×2 LCD screen, and a LED strip progression bar. The back is empty.

The face is now constructed by essential functions of the device

3.  Adding multiple routines with a rotary encoder

We realized if we shuffled all the workout exercises, it would be difficult changing back and forth from a  standing position to a floor exercise. Therefore, this is when we incorporated workout categories for Fredrick’s exercises. Instead of shuffling all the exercises, we would shuffle the exercises in a category. Thus we replaced the button we planned to use before, to a rotary encoder, where the dial part of the rotary encoder would be able to switch between categories, and the button is able to shuffle the exercises in that category.

Using a rotary encoder instead of a button

4. Congratulatory  responses

As we developed our idea of the exercise companion, we wanted to animate the device, even more, to give it more personality. Therefore, we added features where the device would congratulate you at the end of the workout and play a song that you can choose. This detail we hope would allow the user to enjoy the device more.

To see this please look at the video of the interaction under the section, Final Product.

Conclusions and Lessons Learned

During our presentation, one of the verbal comments was that the device had “an infantile appeal” to it.  We thought this was very interesting where we did not mean to make it appear like that at all, in fact, we tried to create the device so that it would appear as a neutral face. Perhaps to make it more friendly, we could animate the expressions and have them more lifelike that way. Looking at our written feedback we also received feedback that stated our device’s appearance was “cute” and they “love the look”. This shows us that our device may have successfully taken a neutral ambiguous look and that is why people are having different reactions to it – it is up to the user to determine the mood of the face. For our case, our user is Fredrick and during our meetings with him to show our device, he enjoyed the appearance. Moving forward, it may be better to include more “faces” to make the device more lively, as pointed out in one of the comments. One of the biggest limitations, however, comes from the LCD display, which has limited ability to display graphics or animation.

In our written comments, more than one person wrote that they were afraid that the user may “game the system” by actively skipping over the exercise right away.  This may be the case, but for our client’s specific situation, he did his exercises daily anyways and only needed this tool to make his routines more interesting. The device is not meant to push him to do the exercises, therefore, this would not have been a problem. If we were to design something for the purpose of helping users exercise, then we could include a mandatory timer for each exercise.

Another comment we received was that “to display the time remaining”. We initially had this feature, but our client declined it as he told us that he wanted to take charge of how long he would spend doing an exercise. He did not want the device to rush him. We think through this project, we learned that designing a device for a one person’s needs makes the device’s functions very specific, therefore, when other’s try to imagine using it, they might not always realize the background information behind it and will not be able to relate to the device as much.

Working remotely was a new experience for all of us, and although it becomes more time-efficient without the scheduling of physical meetings and the making, communicating was more difficult. It was not always very clear what each person was doing and the pace of work was difficult to track as well. The Gantt sheet was good to help us schedule what we needed to accomplish, but what was most effective was actually meeting through the Zoom meetings. Since members of the group did not meet, accountability for accomplishing work was also very individualized, which can be difficult for people facing different situations. This experience made us realize how important physical meetings were. On the other hand, working remotely has turned out to be a better experience than we expected. Even though we cannot have in-person interaction which is important for any activity, we can still communicate effectively through technology if we have the resolution. For example, brainstorming as a group may become harder or different, but with the whiteboard feature on Zoom, it is actually quite easy and engaging.

We really enjoyed talking with the Osher students and making these interesting devices where we didn’t need to think about the cases for multiple users and just one. We hope to take our plans for this project and actually create it in real life sometime in the future and finally be able to hold our hard work in our hands.

Our biggest takeaway from this project would be seeing how important the research and communication with the users are. By talking and understanding our client, Fredrick, our design was changed and guided by his needs. Without this information, the device would not have been as useful and meaningful as it is. This would apply to our future projects as well where we want to focus more time on planning and building a strong foundation of study and research for a design project that I am doing. In addition, we have learned that everyone has their own standard of what a good device or product should be. In the process of designing, we start by envisioning the device in our perspectives. Then it is crucial to communicate with the actual users to find what is working and what is off. Oftentimes there may be features that the designers thought is important but not so by the users, or features that the users value but are not considered by the designers.

Technical Details

Tinkercad link: https://www.tinkercad.com/things/0RXfOLjEqfM

Breadboard Image

This is the annotated breadboard image

Code

/*Exercise Companion
 * Zerui Huo
 * The code makes a robot-shaped device show the exericise names through the selected workout set. It goes to the next execise when it detects sound that has volume
   over the threshold. It also shows reward when the user finishes a group of exercises or the whole set. A newpixel strip shows the progress through the workout.

 *Pin mapping:
  
     pin   | mode   | description
     ------|--------|------------
     6      INPUT     This pin represents the press button on a rotary encoder.
     7      OUTPUT    The pin that controls the Neopixel ligt strip/stick
     8      INPUT     The pin that connects to the microphone that sense any audio input from the user.
     10     OUTPUT    The pin that controls one of the two speakers.
     13     OUTPUT    The pin that controls one of the two speakers.
     A0     INPUT     The pin that represents the rotary turner on the a rotary encoder.
     A3     OUTPUT    Used as digital pin to control LEDS light.
     A4     OUTPUT    Used as digital pin to control LEDS light.
     VCC              The pin for power to go into the Arduino pro mini.Connected to a 9V battery.
     

 *Credit to example codes or reference:
  *The code that generates the custom characters in LCD is written in the help of the website https://maxpromer.github.io/LCD-Character-Creator/.
  *The code for the neopixel light strip is written by refering to the information in the website https://learn.adafruit.com/adafruit-neopixel-uberguide/arduino-library-use. It is a turtorial on how to use the neopixel library.
  

 *Note: In each workout set, exercises requiring similar pose are put into groups. Tree pose, for example, is grouped with other yoga exericise with standing pose.
        The workout sets always have the same groups of exercises,but the sequence of the groups is randomized every time. In addition, the sequence for exerices in
        each group is also randomized.
*/


//Included the needed Libraries
#include <Wire.h>
#include <LiquidCrystal.h>
#include <Adafruit_NeoPixel.h>

//Defining the "audio files" for the exerice names
#define NOTE_C3  131
#define NOTE_CS3 139
#define NOTE_D3  147
#define NOTE_DS3 156
#define NOTE_E3  165
#define NOTE_F3  175
#define NOTE_FS3 185
#define NOTE_G3  196
#define NOTE_GS3 208
#define NOTE_A3  220
#define NOTE_AS3 233
#define NOTE_B3  247
#define NOTE_C4  262
#define NOTE_CS4 277
#define NOTE_D4  294
#define NOTE_DS4 311
#define NOTE_E4  330
#define NOTE_F4  349
#define NOTE_FS4 370
#define NOTE_G4  392
#define NOTE_GS4 415
#define NOTE_A4  440
#define NOTE_AS4 466
#define NOTE_B4  494
#define NOTE_C5  523
#define NOTE_CS5 554
#define NOTE_D5  587
#define NOTE_DS5 622
#define NOTE_E5  659
#define NOTE_F5  698
#define NOTE_FS5 740
#define NOTE_G5  784
#define NOTE_GS5 831
#define NOTE_A5  880
#define NOTE_AS5 932
#define NOTE_B5  988
#define NOTE_C6  1047
#define NOTE_CS6 1109
#define NOTE_D6  1175
#define NOTE_DS6 1245
#define NOTE_E6  1319
#define NOTE_F6  1397
#define NOTE_FS6 1480
#define NOTE_G6  1568
#define NOTE_GS6 1661
#define NOTE_A6  1760
#define NOTE_AS6 1865
#define NOTE_B6  1976
#define NOTE_C7  2093
#define NOTE_CS7 2217
#define NOTE_D7  2349
#define NOTE_DS7 2489
#define NOTE_E7  2637
#define NOTE_F7  2794
#define NOTE_FS7 2960
#define NOTE_G7  3136
#define NOTE_GS7 3322
#define NOTE_A7  3520
#define NOTE_AS7 3729
#define NOTE_B7  3951
#define NOTE_C8  4186
#define NOTE_CS8 4435
#define NOTE_D8  4699
#define NOTE_DS8 4978



//Naming all the pins according to stimulation in TinkerCad
//Remember to change the pin according to your wiring
const int ROTARY_BUTTON = 6;
const int ROTARY_TURNER = A0;
const int MICROPHONE = 8;
const int SPEAKER1 = 10;
const int SPEAKER2 = 13;
const int NEOPIXELSTRIP = 7;
const int LED1 = A3;
const int LED2 = A4;

//Setting the constant ints. Fre stands for the frequency to check a specific timer. Period is the period for some events to last.
const int LED_COUNT = 6;
const int SELECTINGFRE = 100;
const int MICROPHONEFRE = 20;
const int AUDIOPERIOD = 60;
const int STRIPPERIOD = 100;

//Global varibales that change
//Timers for "run without block"
unsigned long selectingTimer = 0;
unsigned long microphoneTimer = 0;
unsigned long audioTimer = 0;
unsigned long stripTimer = 0;

int on_off = 1;
int select = 1;
int cursor = 0;
int row;
int column = 0;
int rowIncrement = 0;
int columnIncrement = 0;
int num_of_groups;

//Lists that store informations
//The set names
String sets[] = {"Yoga Set", "Weights", "Aerobics"};

//The list that tells the row number that a set starts and ends
int set_row[] = {0, 2, 4, 8};

//Example workout sets. The "0" represents a null, and the device will go to the next group of exercises in the set.
String exercise[][3] = {
  {"Jumping Jack Jumping Jack Jumping Jack", "Cobbler Pose", "Abdominal Plank"},
  {"hee", "ha", "yo"},
  {"Tree Pose", "dance", "Upward Facing Dog"},
  {"Cat-cow", "Knee-to-chest", "0"},
  {"Knees-to-chest", "Hip flexor", "Torso rotation"},
  {"Tree pose", "Full body", "Child's Pose"},
  {"Cat-cow", "Crescent warrior", "0"},
  {"Ab plank", "Cobbler Pose", "0"},
};

//Micmic the SD card that stores the audio files of the names of the exercise
int exercise_audio[][3] = {
  {NOTE_C3, NOTE_D3, NOTE_E3},
  {NOTE_F3, NOTE_G3, NOTE_A3},
  {NOTE_B3, NOTE_C4,  NOTE_D4},
  {NOTE_E4, NOTE_F4, NOTE_F5},
  {NOTE_G4, NOTE_A4,NOTE_B4},
  {NOTE_C5, NOTE_D5, NOTE_E5},
  {NOTE_F5, NOTE_G5,  NOTE_A5},
  {NOTE_B5, NOTE_C6, NOTE_D6},
};

//Creating the custom characters for the "face" of the robot
//surprise face
byte SURPRISE_06[] = {
  B00000,
  B00001,
  B00010,
  B00100,
  B01000,
  B01000,
  B01000,
  B01000
};

byte SURPRISE_07[] = {
  B01111,
  B10000,
  B00000,
  B00000,
  B00000,
  B00000,
  B00000,
  B00000
};

byte SURPRISE_08[] = {
  B11110,
  B00001,
  B00000,
  B00000,
  B00000,
  B00000,
  B00000,
  B00000
};

byte SURPRISE_09[] = {
  B00000,
  B10000,
  B01000,
  B00100,
  B00010,
  B00010,
  B00010,
  B00010
};

byte SURPRISE_16[] = {
  B01000,
  B01000,
  B01000,
  B01000,
  B00100,
  B00010,
  B00001,
  B00000
};


byte SURPRISE_17[] = {
  B00000,
  B00000,
  B00000,
  B00000,
  B00000,
  B00000,
  B10000,
  B01111
};

byte SURPRISE_18[] = {
  B00000,
  B00000,
  B00000,
  B00000,
  B00000,
  B00000,
  B00001,
  B11110
};

byte SURPRISE_19[] = {
  B00010,
  B00010,
  B00010,
  B00010,
  B00100,
  B01000,
  B10000,
  B00000
};

//smiley face
byte SMILE_04[] = {
  B00000,
  B00000,
  B00000,
  B00000,
  B00000,
  B00001,
  B00010,
  B00100
};

byte SMILE_05[] = {
  B00000,
  B00000,
  B00000,
  B01000,
  B11000,
  B00100,
  B00010,
  B00001
};


byte SMILE_010[] = {
  B00000,
  B00000,
  B00000,
  B00010,
  B00010,
  B00101,
  B01000,
  B10000
};

byte SMILE_011[] = {
  B00000,
  B00000,
  B00000,
  B00000,
  B00000,
  B00000,
  B10000,
  B01000
};

byte SMILE_16[] = {
  B10000,
  B01000,
  B00100,
  B00010,
  B00001,
  B00000,
  B00000,
  B00000
};

byte SMILE_17[] = {
  B00000,
  B00000,
  B00000,
  B00000,
  B00000,
  B10000,
  B01111,
  B00000
};

byte  SMILE_18[] = {
  B00000,
  B00000,
  B00000,
  B00000,
  B00000,
  B00001,
  B11110,
  B00000
};


byte SMILE_19[] = {
  B00001,
  B00010,
  B00100,
  B01000,
  B10000,
  B00000,
  B00000,
  B00000
};




// generating a random sequence for exerices in each exercise group
int * generate_randnumb_exercise(int min, int length) {
  static int rando[10];
  for (int i = min; i < min + length; i++) {
    rando[i - min] = i;
  }



  randomSeed(digitalRead(A5));
  for (int i = 0; i < 200; i++) {
    int rand1;
    int rand2;
    int old_rand1;
    int old_rand2;
    rand1 = random(0, length - 1);
    rand2 = random(0, length - 1);
    old_rand1 = rando[rand1];
    old_rand2 = rando[rand2];
    rando[rand1] = old_rand2;
    rando[rand2] = old_rand1;
  }
  return rando;
}

// generating a random sequence for exerice group in set1
int * generate_randnumb_set1(int min, int length) {
  static int rando2[10];
  for (int i = min; i < min + length; i++) {
    rando2[i - min] = i;
  }


  randomSeed(digitalRead(A5));
  for (int i = 0; i < 200; i++) {
    int rand1;
    int rand2;
    int old_rand1;
    int old_rand2;
    rand1 = random(0, length - 1);
    rand2 = random(0, length - 1);
    old_rand1 = rando2[rand1];
    old_rand2 = rando2[rand2];
    rando2[rand1] = old_rand2;
    rando2[rand2] = old_rand1;
  }
  return rando2;
}


// generating a random sequence for exerice group in set2
int * generate_randnumb_set2(int min, int length) {
  static int rando3[10];
  for (int i = min; i < min + length; i++) {
    rando3[i - min] = i;
  }



  randomSeed(digitalRead(A5));
  for (int i = 0; i < 200; i++) {
    int rand1;
    int rand2;
    int old_rand1;
    int old_rand2;
    rand1 = random(0, length - 1);
    rand2 = random(0, length - 1);
    old_rand1 = rando3[rand1];
    old_rand2 = rando3[rand2];
    rando3[rand1] = old_rand2;
    rando3[rand2] = old_rand1;
  }
  return rando3;
}


// generating a random sequence for exerice group in set3
int * generate_randnumb_set3(int min, int length) {
  static int rando4[10];
  for (int i = min; i < min + length; i++) {
    rando4[i - min] = i;
  }



  randomSeed(digitalRead(A5));
  for (int i = 0; i < 200; i++) {
    int rand1;
    int rand2;
    int old_rand1;
    int old_rand2;
    rand1 = random(0, length - 1);
    rand2 = random(0, length - 1);
    old_rand1 = rando4[rand1];
    old_rand2 = rando4[rand2];
    rando4[rand1] = old_rand2;
    rando4[rand2] = old_rand1;
  }
  return rando4;
}

//Pointers that refers to the sequence of exercise/exercise groups
int *sequenceExercise = generate_randnumb_exercise(0, 3);
int *sequenceSet1 = generate_randnumb_set1(set_row[0], set_row[1] - set_row[0]);
int *sequenceSet2 = generate_randnumb_set2(set_row[1], set_row[2] - set_row[1]);
int *sequenceSet3 = generate_randnumb_set3(set_row[2], set_row[3] - set_row[2]);

//set up of the lcd
const int rs = 12, en = 11, d4 = 5, d5 = 4, d6 = 3, d7 = 2;
LiquidCrystal lcd(rs, en, d4, d5, d6, d7);

//set up the neopixel strip
Adafruit_NeoPixel strip(LED_COUNT, NEOPIXELSTRIP, NEO_GRB + NEO_KHZ800);
//defining the color for the neopixel strip
uint32_t deep_sky_blue = strip.Color(0, 191, 255);


//Supplemental Functions
//Functions for selecting a workout set
//Determine which set the rotary turner is "pointing at"
int which_set () {
  int error_angle = 30;
  int potenValue = analogRead(ROTARY_TURNER);
  int set = 0;

  //Display the name of set 1 when the potentiometer is at 0 degree
  if (potenValue <= 1023 * error_angle / 300) {
    set = 1;
    return set;
  }


  else if (potenValue >= 512 - 1023 * error_angle / 300 && potenValue <= 512 + 1023 * error_angle / 300) {
    set = 2;
    return set;
  }

  //set 3
  else if (potenValue >= 1023 * (1 - 1 * error_angle / 300)) {
    set = 3;
    return set;
  }
}

//Display the name of the workout set that is currently looking at
void select_set() {
  int buttonState = digitalRead(ROTARY_BUTTON);
  int set;
  set = which_set();
  String text = sets[set - 1];
  int startingColumn = ceil((16 - text.length()) / 2);

  lcd.setCursor(2, 0);
  lcd.print("Select a Set");


  if (on_off == 1) {
    lcd.setCursor(startingColumn, 1);
    lcd.print(text);
  }
  if (on_off == 0) {
    lcd.setCursor(startingColumn, 1);
    lcd.print("               ");
  }

  //When the pressbutton on rotary encoder is pressed
  if (buttonState == LOW) {
    select = 0;

    if (set == 1) {
      row = *sequenceSet1;
      num_of_groups = set_row[1]-set_row[0];
      Serial.println(row);
    }
    if (set == 2) {
      row = *sequenceSet2;
      num_of_groups = set_row[2]-set_row[1];
      Serial.println(row);
    }
    if (set == 3) {
      row = *sequenceSet3;
      num_of_groups = set_row[3]-set_row[2];
      Serial.println(row);
    }

    lcd.clear();
    lcd.setCursor(3, 0);
    lcd.print("Lets Begin!");
    delay(1500);
    lcd.clear();
    //Call the function robot_face to show a smiley face on the lcd
    robot_face("smiley");
    delay(2000);
    lcd.clear();
    audioTimer = millis();
  }
}

//Sumplementary function for display exercise name onto the LCD display
//This function scroll the text when it is too long to display on the 16-digit screen
void scrollLeft(String string, int row) {
  int length = string.length();
  if (millis() - selectingTimer > SELECTINGFRE) {
    if (cursor + 1 <= length ) {
      cursor = cursor + 1;
    }
    if (cursor + 1 > length) {
      cursor = 0;
    }
    selectingTimer = millis();
    lcd.clear();
  }
  String text = string.substring(cursor);
  lcd.setCursor(0, row);
  lcd.print(text);
}

//Function that show the name of the current exercise
void display_text(String string) {
  int length = string.length();
  if (length <= 16) {
    lcd.setCursor(ceil((16 - length) / 2), 0);
    lcd.print(string);
  }

  if (length > 16 && length < 32) {
    int indexofSpace = string.indexOf(" ");
    String firstWord = string.substring(0, indexofSpace);
    String secondWord = string.substring(indexofSpace + 1);

    if (firstWord.length() > 16) {
      scrollLeft(firstWord, 0);
    }
    if (firstWord.length() <= 16) {
      lcd.setCursor(ceil((16 - firstWord.length()) / 2), 0);
      lcd.print(firstWord);
    }

    if (secondWord.length() > 16) {
      scrollLeft(secondWord, 1);
    }
    if (secondWord.length() <= 16) {
      lcd.setCursor(ceil((16 - secondWord.length()) / 2), 1);
      lcd.print(secondWord);
    }
  }
  if (length >= 32) {
    scrollLeft(string, 1);
  }
  
  if (string == "end") {
    lcd.setCursor(ceil((16 - 11) / 2), 0);
    lcd.print("The workout");
    lcd.setCursor(ceil((16 - 11) / 2), 1);
    lcd.print("is finished");
  }
}

//This micmics the function of the mini mp3 player, which plays a audio record of the name of the exercise.
void play_audio(int audio) {
  while (millis() - audioTimer < AUDIOPERIOD) {
  tone(SPEAKER1,audio);
  tone(SPEAKER2,audio);
  }
  noTone(SPEAKER1);
  noTone(SPEAKER2);
}

//Rolling the light on the Neopixel light strip as a reward when a exercise group is finished
void light_up_strip() {
  while (millis() - stripTimer < STRIPPERIOD) {
  int redColor = random(0, 255);
  int greenColor = random(0,255);
  int blueColor = random(0, 255);
  for (int i=0;i < 6; i++) {
  strip.setPixelColor(i,strip.Color(redColor,greenColor,blueColor));
  }
  strip.show();
  }
  strip.clear();
}

//That function that put the custom characters onto the lcd
void robot_face(String face) {
  if (face == "surprise") {
     //Create the custom characters
     lcd.createChar(0, SURPRISE_06);
     lcd.createChar(1, SURPRISE_07);
     lcd.createChar(2, SURPRISE_08);
     lcd.createChar(3, SURPRISE_09);
     lcd.createChar(4, SURPRISE_16);
     lcd.createChar(5, SURPRISE_17);
     lcd.createChar(6, SURPRISE_18);
     lcd.createChar(7, SURPRISE_19);

     //write the characters to their places
     lcd.setCursor(6, 0);
     lcd.write(byte(0));
     lcd.setCursor(7, 0);
     lcd.write(1);
     lcd.setCursor(8, 0);
     lcd.write(2);
     lcd.setCursor(9, 0);
     lcd.write(3);
      lcd.setCursor(6, 1);
     lcd.write(4);
     lcd.setCursor(7, 1);
     lcd.write(5);
     lcd.setCursor(8, 1);
     lcd.write(6);
     lcd.setCursor(9, 1);
     lcd.write(7);
  }

   if (face == "smiley") {
     //Create the custom characters
     lcd.createChar(0,  SMILE_04);
     lcd.createChar(1,  SMILE_05);
     lcd.createChar(2,  SMILE_010);
     lcd.createChar(3,  SMILE_011);
     lcd.createChar(4,  SMILE_16);
     lcd.createChar(5,  SMILE_17);
     lcd.createChar(6,  SMILE_18);
     lcd.createChar(7,  SMILE_19);
    
     //write the characters to their places
     lcd.setCursor(4, 0);
     lcd.write(byte(0));
     lcd.setCursor(5, 0);
     lcd.write(1);
     lcd.setCursor(10, 0);
     lcd.write(2);
     lcd.setCursor(11, 0);
     lcd.write(3);
     lcd.setCursor(6, 1);
     lcd.write(4);
     lcd.setCursor(7, 1);
     lcd.write(5);
     lcd.setCursor(8, 1);
     lcd.write(6);
     lcd.setCursor(9, 1);
     lcd.write(7);
  }
}

//Simple function that turn the led on and off
void ledSwith(String command) {
  if (command == "on"){
    digitalWrite(LED1,HIGH);
    digitalWrite(LED2,HIGH);
  }
  if (command == "off"){
    digitalWrite(LED1,LOW);
    digitalWrite(LED2,LOW);
  }
}

//The function that determines what happens when the workout set is finished
void end() {
  display_text("end");
  delay(2000);
  lcd.clear();
  display_text("Great Job!");
  delay(2000);
  lcd.clear();
  robot_face("smiley");
  int redColor = random(0, 255);
  int greenColor = random(0,255);
  int blueColor = random(0, 255);
  
  //The for loop rolls the lgiht on the LED light strip until the device is powered off
  for (int i=0;i < 6; i++) {
  strip.setPixelColor(i,strip.Color(redColor,greenColor,blueColor));
  strip.show(); 
  delay(400);

    if (i==5) {
      i = -1;
      redColor = random(0, 255);
      greenColor = random(0,255);
      blueColor = random(0, 255);
    }
  }
}



void show_exercise() {
  //Displaying the name of the exercise by feeding the word to function display_text()

  
  //If the selected workout set is set 1
  if (row < set_row[1]) {
    row = *(sequenceSet1 + rowIncrement);
    column = *(sequenceExercise + columnIncrement);
   

    // columnIncrement in  the if-statement should be set to the number of columns in each row
    //The if statement checks if the exercise group is finished
    if (exercise[row][column] == "0" || columnIncrement == 3) {
      rowIncrement = rowIncrement + 1;
      columnIncrement = 0;
      row = *(sequenceSet1 + rowIncrement);
      column = *(sequenceExercise + columnIncrement);

      ledSwith("on");
      robot_face("surprise");
      light_up_strip();
      lcd.clear();
      ledSwith("off");
      audioTimer = millis();
      
      //to check if the set is finished
      if (rowIncrement == set_row[1]) {
        end();
      }
    }
    display_text(exercise[row][column]);
    play_audio(exercise_audio[row][column]);
    
  }

  //If the selected workout set is set 2
  if (row >= set_row[1] && row < set_row[2]) {
    row = *(sequenceSet2 + rowIncrement);
    column = *(sequenceExercise + columnIncrement);
    

    // columnIncrement in  the if-statement should be set to the number of columns in each row
    if (exercise[row][column] == "0" || columnIncrement == 3) {
   
      rowIncrement = rowIncrement + 1;
      columnIncrement = 0;
      row = *(sequenceSet2 + rowIncrement);
      column = *(sequenceExercise + columnIncrement);
      
      ledSwith("on");
      robot_face("surprise");
      light_up_strip();
      lcd.clear();
      ledSwith("off");
      audioTimer = millis();
        
      //to check if the set is finished
      if (rowIncrement == set_row[2] - set_row[1]) {
        end();
      }
    }
    display_text(exercise[row][column]);
    play_audio(exercise_audio[row][column]);
    
  }

  //If the selected workout set is set 3
  if (row >= set_row[2] && row < set_row[3]) {
    row = *(sequenceSet3 + rowIncrement);
    column = *(sequenceExercise + columnIncrement);
   

    // columnIncrement in  the if-statement should be set to the number of columns in each row
    if (exercise[row][column] == "0" || columnIncrement == 3) {

      rowIncrement = rowIncrement + 1;
      columnIncrement = 0;
      row = *(sequenceSet3 + rowIncrement);
      column = *(sequenceExercise + columnIncrement);
      
      robot_face("surprise");
      ledSwith("on");
      light_up_strip();
      lcd.clear();
      ledSwith("off");
      audioTimer = millis();

      
      //to check if the set is finished
      if (rowIncrement == set_row[3] - set_row[2]) {
        end();
      }
    }
    display_text(exercise[row][column]);
    play_audio(exercise_audio[row][column]);
  }
}

//This funtion controls the neopixel strip to show the progress through the set
void show_progress(int num_of_groups) {
  int lightedPixel = (rowIncrement*6)/num_of_groups;
  
  for (int i=0;i < lightedPixel; i++) {
  strip.setPixelColor(i,deep_sky_blue);
  }
  strip.show();
}


void setup() {
  // Setting up the pins
  pinMode(ROTARY_BUTTON, INPUT_PULLUP);
  pinMode(ROTARY_TURNER, INPUT);
  pinMode(SPEAKER1, OUTPUT);
  pinMode(SPEAKER2, OUTPUT);
  pinMode(MICROPHONE,INPUT_PULLUP);
  pinMode(LED1, OUTPUT);
  pinMode(LED2, OUTPUT);

  //Setting up for the lcd screen
  lcd.begin(16, 2);
  lcd.setCursor(2, 0);
  lcd.print("Lets Exercise!");
  delay(1500);
  lcd.clear();

  //Initiate the Neopixel Strip
  strip.begin();
  strip.show();


  //Initiate Serial Print
  Serial.begin(9600);
}

void loop() {
  while (select == 1) {
    select_set();
    //Timer to make the chosen workout set "flash" 
    if (millis() - selectingTimer > SELECTINGFRE) {
      if (on_off == 1) {
        on_off = 0;
      }
      else {
        on_off = 1;
      }
      selectingTimer = millis();
    }
  }

  if (millis() - microphoneTimer > MICROPHONEFRE) {
    int buttonState = digitalRead(MICROPHONE);
    if (buttonState == LOW) {
      audioTimer = millis();
      stripTimer = millis();
      columnIncrement = columnIncrement + 1;
      lcd.clear();
    }
    microphoneTimer = millis();
  }
  show_exercise();
  show_progress(num_of_groups);
}

Schematic and Design Files

Schematic

Exercise Companion.stl

Ultimately, it will reside in the 3D printing room in the back of Hunt A5.

Process

The first point at which I had to make a major decision was in figuring out whether or not I would incorporate sensors for both filaments. For context, the Ultimaker 3 can use 2 filaments when printing. The first is often the build material, which is typically either:

• PLA–PolyLactic Acid, which is basically corn or sugar plastic and is biodegradable.
• ABS–Acrylonitrile Butadiene Styrene, which is commonly used in manufacturing.

The second is often a support material, which is typically either:

• PVA–PolyVinyl Alcohol, which is water soluble plastic.
• Breakaway, which is a plastic meant to be easily removed from the finished part.

Ideally, I would want to include sensors with both, but it turns out that PVA is transparent to infrared. Since the break-beam sensors I planned to use operated with infrared LEDs, they would work fine with the build materials, but not with PVA. They wouldn’t see the strand of PVA. At this point, I either would have needed to make a custom sensor that uses a regular LED and a photoresistor, or order a break-beam sensor that uses visible light. When I am able to return to this project, chances are high that I will just order some sensors that use visible light.

When the build material is low (not triggering the break-beam sensor), the screen will same something like this.

The same goes for the support material.

Here’s what is displayed when all filaments are OK. At this point, I was triggering the sensors with a screwdriver.

Another decision I had to make came as the deadline for the critique was fast approaching. I could either wire up one set of sensors and hopefully have that fully functioning, or try to wire all 4 sets up. This was ultimately my downfall. I never truly made a decision one way or the other, so the result was a severely underdeveloped device that barely worked. The 1 sensor that was wired up didn’t work at the time of critique due to faulty soldering on my part. Then again, this is what happens when you rush.

Discussion

I was relieved that those critiquing my underdeveloped device at least gave me constructive feedback and pointed out the good things about the project. Quite obviously, however, it was a far cry from faultless. Given the state the device was in, I can see why one critic said this:

“Would it have been easier and prone to less error if you made 4 seperate detectors? Then if something goes wrong, you would only need to fix 1 of the printers instead of putting all four sensors off-line.”

While I do agree that it would likely be easier to have 4 separate detectors for the sake of serviceability, it is a little bit wasteful when all 4 detectors could be driven off of 1 Arduino Uno R3 (or a Photon, more on that later).

Another critic said this:

“The project sounds very sophisticated and pretty useful for your job. However, how can you make it be more straightforward for the next users to use that have your job? Make it easier to understand?”

I think the reason it is so difficult to understand is because of how unfinished it is. Once the device is actually finished (hopefully this will be shortly after the COVID-19 pandemic dies down), it should make a lot more sense.

With all of that said, I am quite unhappy with how the project turned out. However, the issues that it has almost entirely trace back to me. In particular, my indecision was my biggest problem, and it led me to waste a lot of time. I think it goes without saying that I did not satisfy my goals for the project.

One truly bone-headed decision I made was using individual wires for each of the pins, rather than being smart and following RZach’s suggestion of using multi-core wire. It would have been the same level of difficulty in terms of soldering, but the external wiring would have been much cleaner. With that in mind, when I am finally able to resume work on this device, the sensors will all use multi-core wire where applicable. Everything will also be wired into a protoboard, rather than a breadboard, so as to make the device more permanent.

One final thing to note when continuing this project is that it will likely be ported to work with a Particle Photon (suggested by Cody Soska and RZach). This will enable me to use IFTTT (if-this-then-that) more easily so that I can have the device send a Slack message to IDeATe staff (Tech Advisors in particular) saying which printer is low on filament and which filament it’s low on. It’s possible to do this with an Arduino, but much easier with a Photon.

Technical Information

code

NOTE: The code you are about to see is very unfinished. That said, it does compile, and it properly reads the sensors (when they’re wired correctly). Once I am able to wire the device up correctly, I will update this post with revised code, both for the final Arduino version, as well as the Photon version. Regardless, here’s the code as it was during critique:

/*
   Project 2
   Seth Geiser (sgeiser)

   Collaboration: None

   Challenge: The biggest challenge was getting the sensor wired up
   correctly. I did cook one. :/

   Next time: I don't know... keep going?

   Description: The purpose is to create a set of sensors that can
   detect when a 3D printer is running out of filament. Two break-
   beam sensors are used for this purpose -- one for the primary
   (build) material and another for the secondary (support)
   material. When material is blocking the sensor, the sensor will
   read false, meaning that it is not yet critically low. When
   material is no longer blocking the sensor, the sensor will read
   true, meaning that the material is critically low and should be
   replaced with a new reel. At the very least, the device will
   sound an alarm (not implemented) and display which printer is
   low on filament and which filament it is low on (on the I2C
   LCD). The final version will do this and send a Slack message
   using IFTTT to IDeATe Tech Advisors.

   Pin mapping:

   pin  | mode  | description
   -----|-------|------------
   2      input   break-beam sensor 1 (build material)
   3      input   break-beam sensor 2 (support material)
   SDA/SCL        display

*/

#include <Wire.h>
#include <LiquidCrystal_I2C.h>

/*
   Create an LCD display object called "screen" with I2C address 0x27
   which is 20 columns wide and 4 rows tall.
*/

LiquidCrystal_I2C screen(0x27, 20, 4);

const int BUILDSENSOR = 2;
const int SUPPORTSENSOR = 3;

unsigned long timer = 0; // variable for timing
const int INTERVAL = 1000; // milliseconds between updates


void setup() {
  pinMode(BUILDSENSOR, INPUT);
  pinMode(SUPPORTSENSOR, INPUT);

  screen.init();
  screen.backlight();
  screen.home();
  Serial.begin(9600);
}

void loop() {
  bool buildCritical = digitalRead(2);
  bool supportCritical = false;
  /*
     would be digitalRead(3), but sensor was not wired up in time for
     critique, so I left it at false to eliminate any chances of an
     ambiguous reading.
  */
  // updating the LCD output
  if (millis() >= timer) {
    if (!buildCritical) {
      screen.setCursor(0, 0);
      screen.print((String) "All filaments OK.");
      screen.setCursor(0, 1);
      screen.print((String) "Support sensor");
      screen.setCursor(0, 2);
      screen.print((String) "is missing!");
    }
    else if (buildCritical) {
      screen.setCursor(0, 0);
      screen.print((String) "Build material  ");
      screen.setCursor(0, 1);
      screen.print((String) "is low!");
    }
    else if (supportCritical) {
      screen.setCursor(0, 0);
      screen.print((String) "Support material");
      screen.setCursor(0, 1);
      screen.print((String) "is low!");
    }
    timer = millis() + INTERVAL; // and update timer
  }

}

schematic

Unlike the code, the schematic for this device shouldn’t change too much, even when transitioning from an Arduino Uno R3 to a Particle Photon — some pin numbers may change, but that’s about it. This was made using Fritzing.

All four sets of sensors are wired up identically.