What We Built

We created a braille alarm clock. The alarm clock is fully functional by someone who is both without sight and hard of hearing. The user can check and reset the current time, as well as set the time for when they’d like their alarm to go off, and it will reliably do so.

Final clock with bed shaker and original alarm clock on presentation day

Top interface of clock with 12 red buttons in a circle for the clock, and 2 red buttons on the left (alarm and clock), and 2 green buttons (set and clear)

Right side of prototype with solenoid and rocker switch

Back of clock with acrylic flower details and power plug

Additional flower acrylic details at the front

Alycia testing out our final device. (Seated to the right of the a table is Alycia, a middle-aged woman with long hair. She is pressing buttons on a pink box that acts as a clock with brailled on it)

Narrative Sketch

Alicia has a big day tomorrow and wants to get an early start in the morning. She goes to her alarm clock, presses the arcade button marked Alarm and then shifts her hand to the clock buttons. She presses the fifth button, and then the sixth button: 5:30.She then presses a different button market Set. Just to double check that she didn’t make a mistake, she presses the Alarm button once more. Now the exciting part… she shifts her hand to the right wall of the clock, feeling for a circle opening with felt under it. Two of her fingers lightly rest on the felt and she waits. Tap, Tap, Tap, Tap, Tap. 5. There’s a short pause. Tap, Tap, Tap. 3. There’s a much pause. 0. A quick Tap-Tap. Knowing that the end of the sequence is always marked by the Tap-Tap, she goes to bed content.

Come morning time, the bed shaker violently rocks her bed waking her up. She gets out of bed, goes to the alarm clock and switches the switch from ON to OFF. The bed shaker stops immediately. She wants to check the time just to ensure she didn’t miss anything in her day, so she presses Clock and moves her fingers to the felt again. 5 taps, 3 taps, and 1 tap. 5:31am. She nods with satisfaction and starts her day.

How We Got Here (Prototype and Process)

This prototype was designed to help answer the design question: How do we make an alarm clock for a blind person that is hard of hearing?

Prototype

First meeting discussion possible ideas

Second meeting discussing clock ideas

Final 3 ideas for prototyping

Prototype 1: Two clocks where each number is a button. The first clock would be for setting the hours, and the second clock would be for setting the minutes.

Prototype 2: A finger slider, where you put your finger on a slider and the top of your finger would slide across different braille numbers. When you got to the number you wanted first, you would press set, and that would set the hours. Then for the minutes, you would slide to the number you wanted and that would set the minutes.

Prototype 3: The third idea is similar to a rotary phone where you would put your finger in a small hole and circle around all the numbers to the number you were looking for. In each number spot would be braille that the user can read, and they would use the rotary system to set the hours and then minutes of their alarm.

Showing Alycia the prototypes on prototype day

Our 3 prototype interfaces

testing prototypes with Alycia

Alycia showing us how to make braille on dymo tape
(Alycia is using a poking device and a braille grid to press indentations onto sticky tape)

All of our prototypes were feasible, yet how we moved forward was solely to the discretion of our client and what her personal preference was in terms of the alarm. She chose Prototype 1, where the interface is all buttons in a circle.

We were a bit surprised that the client chose the option she did. When she went over her reasoning, she explained that practicality and ease-of-use in the long term were her main decision factors. The other ideas were fun and exciting, but there’s much less that could go wrong with simple buttons. She also gave feedback that was eye-opening in the sense that we were underestimating her skills ability to a great extent. Things such as only needing one button clock interface and not two, and needing less braille options, her intuit knowledge that could help her use the alarm with ease. Initially when designing and speaking amongst each other, we were approaching the prototype as if we had just lost our vision and were encountering the world for the first time. However, after talking with Alyicia, we were able to revise our prototypes and make them not only braille friendly, but also extremely intuitive to her needs.

Feedback we took from the critique was to make it only one clock interface, as well as to change the button size. Alicia opted for smaller buttons around the clock as those would decrease the overall size of the alarm. She also mentioned the need for there to be ample space around each button as that would ensure there was enough room for her dymo tape to stick on. Thus, we made sure to account for the braille attachments and their length and width in each button design. She also told us she had no need for a Snooze button, nor did she need an option to set any time more than the multiples of five. This type of feedback was pivotal to our further discussion of how to implement the alarm clock successfully.

Something we hadn’t even considered before she brought it up, was being able to tell the normal time of the day using the alarm clock. She informed us this was normal practice on her existing braille alarm clock that didn’t work that well. Initially we chose not to focus on this as the clock module should have been able to reliably keep the time and there would be no reason to doubt it. However, after much discussion as a team and with our professor, we reimplemented it as a feature.

Process

Deciding on how to improve prototype after showing to Alycia

Final interface design for the clock

Lasercut plywood to build clock

Painting the plywood

The programmable portion of the alarm clock was based on a finite state machine, a concept that David was familiar with from his ECE studies. This style of design was chosen because an alarm clock enters through many states during its usage, and transitioning between states based on inputs (and generating outputs at the right time) are all central ideas to an FSM. The final product ended up with seven states, and it worked extremely well- it allowed for coding details much easier, as it helped partition up the overall complicated structure into smaller, more manageable pieces. That being said, coding did have its fair share of difficulties. Converting a button into a toggle-able input was surprisingly complex; it involved debouncing a button to prevent erroneous multiple readings upon press and release of the button, as well as checking to make sure when a button is pressed and held (and not causing the incorrect interpretation of multiple button presses). The code also involved understanding a RTC (real-time clock) library, which had its own hidden difficulties (we had spent nearly an hour panicking as to why the clock was not working until we realized we hadn’t written the code to actually START the clock!). With respect to the Gantt chart, there was some divergence from the original plan, because testing the code proved to be more reliant upon the hardware it worked with- for example, it was impossible to check if anything related to the clock was working if there was no clock module, which we did not receive until much later than anticipated. Nonetheless, the code was still completed and thoroughly debugged.

All wiring of the clock

Putting all components into the fingerjoint base

There were a few tricky components to the circuitry, such as the solenoid and the interfacing of the bed shaker and its power supply. The solenoid needed to be powered directly by a 12VDC power supply, yet it had to be controlled by the Arduino; as such, a transistor circuit was used to act as a “switch” that the Arduino could control to allow for power to the solenoid. Interfacing with the bed shaker proved to be very complicated, due to the strange peculiarity that it ran off of 9VAC. Creating a switch for 9V, being alternating current as well, was rather difficult, since that implied it would require a power supply that could not be unified in any way with the Arduino (unlike the solenoid, whose 12VDC power supply could also be used for the Arduino). Controlling an AC power supply also required the use of a relay, which was in essence, a switch for AC power supplies, similar to the transistor. The relay proved to also be troublesome in that it would draw too much current from the Arduino should it be directly connected, and that meant that another transistor circuit was needed for the relay- in essence, a switch for the switch. It was certainly a confusing set of circuitry- a memorable moment was when the transistor was actually fried due to an incorrect wiring, which then caused subsequent correct wirings to not work as well.

Conclusions and Lessons Learned 

We learned throughout the process that planning was very important to ensure that we could get all the different parts done well. Since we each had different responsibilities, it was important that we communicated our schedules well and adjusted our work plan as needed so that everyone in the group could work on their responsibilities. Especially for our code-heavy project, we found that it was always better to test early and test more. There would often be small edge cases that would mess with how our clock worked. Changes also needed to be made physically, such as adding another button or changing where the buttons were placed, which would help make using the device easier for Alycia. Additionally, we realized that we could make the best device only if we knew what the client actually needed and wanted. Rather than making assumptions, we always made sure to ask Alycia on what she thought would work the best for her as well as what she wanted from the project. There were times when her responses reassured our progress, and other where we were surprised by what she said and would have to rethink our ideas. In the end, we were grateful that Alycia was very cooperative and responsive to us during this project as we were able to produce a product that she liked as well. 

A common, but important piece of advice that was given was that “Though it’s for a blind person, for presentations sake, there could have been some visual labels.” This would allow for those who were NOT Alycia (eg. caretakers, friends, etc.) to be able to understand what was what on the clock. This would be a simple change, but extremely necessary, since as of now, we only know which buttons are what simply because we are the creators. There were also comments on the alarm clock having “other features that many clocks have (i.e., snooze).” This was also a fair point, and was considered by our team; given more time, this definitely could have been done. While we considered the situation of having too many inputs and causing confusion rather than more features, simple thoughts like “Different shape on the “12” button” to help find/distinguish the clock buttons easier would be useful and helpful to implement.

Technical Details

Schematic and Block Diagram

Code

/* Project Title: Tactile Alarm Clock
 * Creator: David Wu, Noni Shelton, and Catherine Liu
 * 
 * This code is similar to a finite state machine for
 * an alarm clock. There are 12 button inputs for the 12 
 * numbers on the clock. There are 4 other button inputs 
 * for miscellaneous purposes, being the alarm, clock, 
 * set, and clear buttons. The 16 button inputs are each 
 * involved in 2 arrays to facilitate checking for button
 * presses and button holding. An on/off input is used to 
 * determine if the alarm is turned on or not. An output signal
 * to a solenoid allows it to tap the time in an specified, 
 * encoded manner. The code outputs its alarm signal at a certain
 * pin, and for this project, this is intended to connect the
 * bed shaker to its power supply. 
 * 
 * Pin Mapping:
 * Clock 12 : 12    Clock 6  : 6
 * Clock 1  : 13    Clock 7  : 7
 * Clock 2  : 2     Clock 8  : 8
 * Clock 3  : 3     Clock 9  : 9
 * Clock 4  : 4     Clock 10 : 10
 * Clock 5  : 5     Clock 11 : 11
 * Doing Alarm               : 27
 * Doing Clock               : 26
 * Set                       : 24
 * Clear                     : 22
 * Alarm On/Off              : 28
 * Solenoid                  : 30
 * Alarm output (bed shaker) : 33
 * 
 */

#include <Wire.h>
#include "RTClib.h"

RTC_DS3231 rtc;

// clock 12 inputs
const int CLOCKPIN0 = 12;
const int CLOCKPIN1 = 13;
const int CLOCKPIN2 = 2;
const int CLOCKPIN3 = 3;
const int CLOCKPIN4 = 4;
const int CLOCKPIN5 = 5;
const int CLOCKPIN6 = 6;
const int CLOCKPIN7 = 7;
const int CLOCKPIN8 = 8;
const int CLOCKPIN9 = 9;
const int CLOCKPIN10 = 10;
const int CLOCKPIN11 = 11;

// the miscellaneous buttons
const int DOING_ALARMPIN = 27;
const int DOING_CLOCKPIN = 26;
const int SETPIN = 24;
const int CLEARPIN = 22;

// alarm on/off
const int ON_OFFPIN = 28;

// outputs
const int SOLEPIN = 30;
const int BEDSHAKERPIN = 33;

// state management
int currState = 0;
int nextState = 0;
const int defaultState = 0;
const int alarmState = 1;
const int clockState = 2;
const int setAlarmHourState = 3;
const int setClockHourState = 4;
const int setAlarmMinState = 5;
const int setClockMinState = 6;
const int alarmTriggered = 7;

// clock 12 inputs
int clockInput[12];
int clockPressed[12];

// the miscellaneous buttons
// misc inputs: doing_alarm, doing_clock, set, clear
int miscInput[4]; 
int miscPressed[4];

// alarm on/off
bool isOn = true;

// time management
// arbitrary default setting when clock is first turned on
const int DEFAULT_HOUR = 7;
int alarmHour = DEFAULT_HOUR;
int alarmMin = 0;
int tempHour = DEFAULT_HOUR;
int tempMin = 0;

void setup() {
  pinMode(CLOCKPIN0, INPUT);
  pinMode(CLOCKPIN1, INPUT);
  pinMode(CLOCKPIN2, INPUT);
  pinMode(CLOCKPIN3, INPUT);
  pinMode(CLOCKPIN4, INPUT);
  pinMode(CLOCKPIN5, INPUT);
  pinMode(CLOCKPIN6, INPUT);
  pinMode(CLOCKPIN7, INPUT);
  pinMode(CLOCKPIN8, INPUT);
  pinMode(CLOCKPIN9, INPUT);
  pinMode(CLOCKPIN10, INPUT);
  pinMode(CLOCKPIN11, INPUT);
  pinMode(DOING_ALARMPIN, INPUT);
  pinMode(DOING_CLOCKPIN, INPUT);
  pinMode(SETPIN, INPUT);
  pinMode(CLEARPIN, INPUT);
  pinMode(ON_OFFPIN, INPUT);
  pinMode(SOLEPIN, OUTPUT);
  pinMode(BEDSHAKERPIN, OUTPUT);

  // ensure rtc is found and we can start
  if (!rtc.begin()) {
    while (1);
  }

  // initialize an arbitrary time (12:34)
  // rtc.adjust(DateTime(F(__DATE__), F(__TIME__)));
  rtc.adjust(DateTime(2000, 1, 1, 12, 34, 0));
  Serial.begin(9600);
}

void loop() {
  // clock 12 inputs
  clockInput[0] = digitalRead(CLOCKPIN0);
  clockInput[1] = digitalRead(CLOCKPIN1);
  clockInput[2] = digitalRead(CLOCKPIN2);
  clockInput[3] = digitalRead(CLOCKPIN3);
  clockInput[4] = digitalRead(CLOCKPIN4);
  clockInput[5] = digitalRead(CLOCKPIN5);
  clockInput[6] = digitalRead(CLOCKPIN6);
  clockInput[7] = digitalRead(CLOCKPIN7);
  clockInput[8] = digitalRead(CLOCKPIN8);
  clockInput[9] = digitalRead(CLOCKPIN9);
  clockInput[10] = digitalRead(CLOCKPIN10);
  clockInput[11] = digitalRead(CLOCKPIN11);

  // misc inputs: doing_alarm, doing_clock, set, clear
  miscInput[0] = !digitalRead(DOING_ALARMPIN); // normally closed button
  miscInput[1] = !digitalRead(DOING_CLOCKPIN); // normally closed button
  miscInput[2] = digitalRead(SETPIN); 
  miscInput[3] = digitalRead(CLEARPIN); 
  
  // alarm on/off
  isOn = digitalRead(ON_OFFPIN);

  // find if/which clockInput was pressed
  int trigClockButton = -1;
  for (int i = 0; i < 12; i++) {
    if (clockInput[i] == 1) {
      trigClockButton = i;
    }
  }

  // find if/which miscInput was pressed
  int trigMiscButton = -1;
  for (int i = 0; i < 4; i++) {
    if (miscInput[i] == 1) {
      trigMiscButton = i;
    }
  }    

  // when to start output
  if (isOn) {
    DateTime now = rtc.now();
    if ((convert12Hour(now.hour()) == alarmHour) &&
        (now.minute() == alarmMin)  &&
        (now.second() < 5)) {
       currState = alarmTriggered;
    }
  }

  // figure out next state and output
  if (currState == alarmTriggered) {
    if(!isOn) {
      digitalWrite(BEDSHAKERPIN, LOW);
      nextState = defaultState;     
    }
    else {
      digitalWrite(BEDSHAKERPIN, HIGH);
      nextState = alarmTriggered;
    }
  }
  else if (trigMiscButton != -1)
    handlePressedMiscButton(trigMiscButton);
  else if (trigClockButton != -1) 
    handlePressedClockButton(trigClockButton);
  else 
    nextState = currState;
    
  // update unpressed buttons
  // when a clockInput is unpressed and was pressed before
  for (int i = 0; i < 12; i++) {
    if (!clockInput[i] && clockPressed[i]) {
      delay(30);
    }
  }
  // when a miscInput is unpressed and was pressed before
  for (int i = 0; i < 4; i++) {
    if (!miscInput[i] && miscPressed[i]) {
      delay(30);
    }
  }  
  
  //update info
  clockPressed[0] = clockInput[0];
  clockPressed[1] = clockInput[1];
  clockPressed[2] = clockInput[2];
  clockPressed[3] = clockInput[3];
  clockPressed[4] = clockInput[4];
  clockPressed[5] = clockInput[5];
  clockPressed[6] = clockInput[6];
  clockPressed[7] = clockInput[7];
  clockPressed[8] = clockInput[8];
  clockPressed[9] = clockInput[9];
  clockPressed[10] = clockInput[10];
  clockPressed[11] = clockInput[11];

  miscPressed[0] = miscInput[0];
  miscPressed[1] = miscInput[1];
  miscPressed[2] = miscInput[2];
  miscPressed[3] = miscInput[3]; 
  
  currState = nextState;
}

void handlePressedClockButton (int index) {
  // when the clockInput is pressed and was not pressed before
  if (!clockPressed[index]) {
    delay(30);
    switch (currState) {
      // if alarm is triggered, ignore everything
      case alarmTriggered:
        break;
      case alarmState:
        tempHour = index;
        if (tempHour == 0) tempHour = 12;
        nextState = setAlarmHourState;
        break;
      case clockState:
        tempHour = index;
        if (tempHour == 0) tempHour = 12;
        nextState = setClockHourState;
        break;
      case setAlarmHourState:
        tempMin = index * 5;
        nextState = setAlarmMinState;
        break;
      case setClockHourState:
        tempMin = index * 5;
        nextState = setClockMinState;
        break;
      default: 
        nextState = currState;
        break;
    }
  }
}

void handlePressedMiscButton (int index) {
  // misc inputs: doing_alarm, doing_clock, set, clear
  // when the miscInput is pressed and was not pressed before
  if (!miscPressed[index]) {
    delay(30);
    // if alarm is triggered, ignore everything
    if (currState == alarmTriggered) {
      nextState = alarmTriggered;
    }
    // doing_alarm
    else if (index == 0) {
      solenoidTapping(alarmHour, alarmMin);
      nextState = alarmState;
    }
    // doing_clock
    else if (index == 1) {
      DateTime now = rtc.now();
      solenoidTapping(convert12Hour(now.hour()), now.minute());
      nextState = clockState;
    }
    else {
      switch (currState) {
        case alarmState:
        case setAlarmHourState:
          if (index == 2) {
            // not complete inputs
            tempHour = DEFAULT_HOUR;
            tempMin = 0;
            nextState = defaultState;
          }
          else if (index == 3) {
            tempHour = DEFAULT_HOUR;
            tempMin = 0;
            nextState = alarmState;
          }
          break;
        case clockState:
        case setClockHourState:
          if (index == 2) {
            // not complete inputs
            tempHour = DEFAULT_HOUR;
            tempMin = 0;
            nextState = defaultState;
          }
          else if (index == 3) {
            tempHour = DEFAULT_HOUR;
            tempMin = 0;
            nextState = clockState;
          }
          break;
        case setAlarmMinState:
          if (index == 2) {
            alarmHour = tempHour;
            alarmMin = tempMin;
            nextState = defaultState;
          }
          else if (index == 3) {
            tempHour = DEFAULT_HOUR;
            tempMin = 0;
            nextState = alarmState;
          }
          break;
        case setClockMinState:
          if (index == 2) {
            // DateTime(YYYY, M, D, h, m, s)
            rtc.adjust(DateTime(2000, 1, 1, tempHour, tempMin, 0));
            nextState = defaultState;
          }
          else if (index == 3) {
            tempHour = DEFAULT_HOUR;
            tempMin = 0;
            nextState = clockState;
          }
          break;
        default: 
          nextState = currState;
          break;
      }
    }
  }
}

// Taps: Hours, Tens place of Mins, Ones place of Mins
// Ex. 9:47 is 9 taps, 4 taps, 7 taps
void solenoidTapping (int timeHour, int timeMinute) {
  for (int i = 0; i < timeHour; i++) {
    digitalWrite(SOLEPIN, HIGH);
    delay(50);
    digitalWrite(SOLEPIN, LOW);
    delay(450);
  }
  delay(2000);
  int minuteTens = timeMinute / 10;
  int minuteOnes = timeMinute % 10;
  for (int i = 0; i < minuteTens; i++) {
    digitalWrite(SOLEPIN, HIGH);
    delay(50);
    digitalWrite(SOLEPIN, LOW);
    delay(450);
  }
  delay(2000);
  for (int i = 0; i < minuteOnes; i++) {
    digitalWrite(SOLEPIN, HIGH);
    delay(50);
    digitalWrite(SOLEPIN, LOW);
    delay(450);
  }
  delay(2000);
  // a quick TAP-TAP to signal done tapping
  digitalWrite(SOLEPIN, HIGH);
  delay(50);
  digitalWrite(SOLEPIN, LOW);
  delay(100);
  digitalWrite(SOLEPIN, HIGH);
  delay(50);
  digitalWrite(SOLEPIN, LOW);
  delay(100);
}

// converts hours from 24-hour format to 12-hour format
int convert12Hour(int timeHour) {
  // later half of day
  if (timeHour > 12) 
    return timeHour - 12;
  // midnight
  else if (timeHour == 0)
    return 12;
  else
    return timeHour;
}

The topic of our final project is to design and build a device for our client with physical difficulties. We get to understand our client’s needs by interviewing and project progress meetings during the 6-week project period and keep improving our product according to information we got from conversations and feedback from the client. Our client is Haleigh, and here is the documentation of our first interview: Interview with Haleigh. Haleigh has difficulties moving and twisting her hands and fingers freely, and she doesn’t have enough grip to hold utensils as well as paintbrushes; she loves painting. Based on this information, we built a wearable device that can hold and rotate utensils for her.

2. Final Product

The wearable device we built is a wristband that carries the electronics and a servo motor with a connector to hold and drive different utensils. We crocheted the wristband with light purple yarn to ensure it’s light-weighted and comfortable for our client; and yes, purple is Haleigh’s favorite color. We also choose Velcro to tighten and loosen the wristband thus reducing the wearing difficulty. The utensils and pastel holder are designed, and 3D printed based on our client’s preferred position and rotational angles. To make the control most uncomplicated, we created a discrete button to trigger the utensil rotation, so Haleigh can use one of her hands to use a utensil and control the rotation with another hand.

Utensil Holder with A Spoon

Utensil Holder with A Fork

Whenever Haleigh is, for example, eating a meal, her mother would easily be able to attach the velcro wrist straps on the device and then serve her the food. While before Haleigh mostly ate with her hand, which is still difficult due to her limited wrist movement, with the device she would able to relax more. She would be able to change between the spoon or fork fixtures depending on what she’s eating and be able to have her meal mostly by herself if she so chose to. Every time she hits the button, the servo motor would consistently rotate the utensil towards the food and then towards her, allowing her to eat with more ease.

3. Prototypes and Modifications

3.1 Prototyping

After the first interview, we proposed three essential question for our project:

1. Wearable part – How to make the wristband easy to ware, comfortable, but also sturdy enough to hold devices and utensils;
2. Electronic part – How to select a proper sensor to convert the change of pose to some data, and how the data can use to control the movement of utensils;
3. Utensil part – How to create a universal connector that fits different utensils and connect them.

We discussed these questions and came up with an initial idea sketch:

Prototyping Sketch

Based on our client’s way of holding things and her grip, we proposed a glove-like design with multiple supports: writs, palm, and curlicue for the wristband. Such a method helps to average dispense the gravity of devices and electronics to the whole hand; the wristband could be knitted.

Prototyping – Wristband

For the utensil, we first designed a box-like holder for the disposable tableware – the user can just stab the tail of utensils into the holder, and the holder could fix the utensil by friction and magnetism. However, because of lacking consideration of the size of the magnets, we failed to install magnets to the utensils. Still, we found that the friction could be enough to hold every tableware (made of plastic, not metal).

Prototyping – Utensil Holder

At the end of the prototyping phase, we put parts together and created a rough version of our utensil holder.

The First Prototype

The prototype enables us to check the feasibility of our design. The fun part is that all three parts – electronics, wristband, and utensil holder – work somewhat well as a whole; the wristband can hold the electronics, the electronics can drive the utensils, and the utensil and its hold aren’t too heavy for the servo. Then we brought this initial product to the first critic presentation to get more feedback about potential improvements to our design.

We got very constructive and helpful suggestions from out client and critics during the prototype presentation:

1. Wristband: The wristband can be knitted with thinner yarn, making the band lighter and better ventilated. It’s better to make the wristband cover the whole back of the hand to avoid any hard part making the hand uncomfortable. We can still use a cushion to support the device, as the knitting part is too soft to mount electronics and servo.
2. Utensils: The angle of utensils should be well considered because now we are using the utensil differently: utensils like forks and spoons should somewhat curve towards the mouth. So, we should consider designing utensils instead of using disposable ones.
3. Electronics: The accelerometer is too sensitive to the gesture changes, which might lead to unwilling utensil rotations. We could go with more accessible solutions like using a button to control spoon and fork motions. We could separate the button from the wristband so that our client could use the product more accessible. We could establish wireless communications between the button and electronics on the wristband.

3.2 Interim Modifications

Based on the feedback from the prototype critic, we redesigned the wristband, electronics, and utensils (and the holder). We then invite our client to the Hunt Library again to see if our modifications fit her gestures and needs.

We did a significant improvement on lightness and softness for the wristband part. Still, it’s not sturdy enough – it’s not perfectly wrapping around the hand, so the device on it might move around unwillingly and frequently. So we decide to redesign the belt to make tighten the wristband.

Interim Modification – Wearing the Wristband

Interim Modification – the Wristband Front

Interim Modification – the Wristband Back

The button is working pretty well; the cap is large enough for our client, and the communication between the button and utensil servo seems established, although disconnection happens sometimes. The next step for the button is to design a container for it.

Interim Modification – the Control Button

Interim Modification – the Pastel Holder Angle

3.3 Finalize the Product

With the interim modifications, we used our last week to make final improvements to our design. We mounted electronics onto the wristband, assembled things, did some final tests, and finished the project.

Final Modifications

Utensil Holder Redesign

Final Product

4. Conclusion and Lesson Learned

From the final critique, there were many things that we discussed with both the reviewers and with Haleigh that we could look at to further improve the project. For instance, one point that one of the reviewers brought up with us and on the written feedback sheet was how “[we] could have pushed further with liquid and food protection – a removable, washable cover would have helped polish [it] off”. Another reviewer had also commented that “the 3d printed utensils should be reconsidered” as “the material may not be entirely safe for use with foods”. While the food safety of the project was something that we had considered, given the form of our project and how we had to go about making the specific parts and their shapes, it was difficult to find a way to implement them given our budget and resources. A solution to this that we wish we looked more into was trying to make primarily holders that would be able to perform the same task but with any utensil the same way we made the universal writing implement holder. Another reviewer brought up how it would be better “if the wrist piece had a little sleeve that could contain the electronic pieces”. This is an easy solution which we feel we could accomplish with either another piece of the foam that there currently is or with an additional crochet piece, either of which could be attached with velcro. Another one of the comments that were made by the reviewers was that they were “worried [the device] will turn while on her wrist”, which is something we think that we addressed in our meetings with Haleigh and the customizations that we were able to make to make it as suited to her as possible. 

Overall, despite the numerous points for reconsideration that there are to be considered in evaluating our project, we still were able to learn a lot through this experience of working with a person with a disability. Designing for another client as opposed to ourselves, it was very interesting going about the process of taking into consideration a different set of needs and restrictions which are outside of our own experiences. It surprised us how we had to translate the movements and abilities based on how Haleigh would be able to approach the same tasks. It was interesting to re-evaluate the way that we did something that is almost second nature to us, like eating with a fork and spoon and how these motions can be applied to the way Haleigh can move.

Looking at the project overall, we would go about choosing different materials for the attachments and also consider how to go about making the attachments more universal. Given an unlimited amount of time and resources, our main priority would be to making the attachments food safe and try increase the flexibility of the structure for both comfort and lightening the weight.

5. Technical Details

/*
  Utensil Holder
*/
/*
  This code is for the button, which enables it to remotely control the motor
*/
/*
  Hardware Pinout Connection
  Arduino Uno        nRF24L01
       3.3V ---------- VCC
       GND ----------- GND
       7  ------------ CE
       8  ------------ CSN
       11 ------------ MOSI
       12 ------------ MISO
       13 ------------ SCK
________________________________________
  Arduino Uno         Button
       GND ---------- negative
       2  ----------- positive
*/

#include <Wire.h>
#include <SPI.h>
#include <nRF24L01.h>
#include <RF24.h>

const int buttonPin = 2; // the number of the pushbutton pin
int buttonState = 0;
const byte address[6] = "00001";

RF24 radio(7, 8); // CE, CSN

void setup() {
  pinMode(buttonPin, INPUT_PULLUP);
  Serial.begin(9600);
  Wire.begin();
  radio.begin();
  radio.openWritingPipe(address);
  radio.setPALevel(RF24_PA_MIN);
  radio.stopListening();
}

void loop() {
  buttonState = digitalRead(buttonPin);
  Serial.println(buttonState);
  radio.write(&buttonState, sizeof(buttonState));
}

/*
  Utensil Holder
*/
/*
  This code is for the motor, which enables it to be controlled by the button
*/
/*
  Hardware Pinout Connection
  Arduino Pro Micro        nRF24L01
          VCC -------------- VCC
          GND -------------- GND
          9  --------------- CE
          10  -------------- CSN
          16 --------------- MOSI
          14 --------------- MISO
          15 --------------- SCK
________________________________________
  Arduino Pro Micro       Servo Motor
          VCC -------------- VCC
          GND -------------- GND
          8  --------------- signal
*/

#include <Servo.h>
#include <Wire.h>
#include <SPI.h>
#include <nRF24L01.h>
#include <RF24.h>

int buttonState = 0;
int tracker = 1; //tracking which direction we want the motor to rotate
int btn_prev = buttonState; //track the last buttonState to avoid multiple count of signal
const byte address[6] = "00001";

RF24 radio(9, 10); // CE, CSN
Servo myservo;

void setup() {
  pinMode(buttonPin, INPUT_PULLUP);
  myservo.attach(8);
  Serial.begin(9600);
  Wire.begin();
  radio.begin();
  radio.openReadingPipe(0, address);
  radio.setPALevel(RF24_PA_MIN);
  radio.startListening();
  btn_prev = 1;
  myservo.write(90);
}

void loop() {
  if (radio.available())
  {
    radio.read(&buttonState, sizeof(buttonState));
    Serial.println(buttonState);
    if (buttonState == LOW && btn_prev == 1) { 
      if (tracker == 1) {
        for (double pos = 90; pos <= 180; pos += 0.5) { //rotate from 90 to 180 degrees
          myservo.write(pos);              
          delay(15);  // waits 15ms for the servo to reach the position
        }
        tracker = -tracker;
      }
      else {
        for (double pos = 180; pos >= 90; pos -= 0.5) { //rotate from 180 to 90 degrees
          myservo.write(pos);              
          delay(15);
        }
        tracker = -tracker;
      }
    }  
  }  
  btn_prev = buttonState;
}

What We Built:

Our project is designed to allow our client Allie to type a price into a keypad and then have the amount of money they need to be dispensed to them as well as the amount of change left over displayed. This cash dispenser can be attached to our client’s wheelchair. There is also a reload mechanism that allows our client to reload the machine with money. Our specific device was designed to dispense $10 and $20 bills. 

A general view of our machine. It is a pink box with two sets of money rollers inside. Each roller compose of two tubes covered and connected by a black fabric with two motors attached to the right side.

Infrared Sensors are depicted on the left and Infrared Detectors are shown on the right. These pair of sensors get blocked by the bills dispensed by the module and once the bill stops blocking the infrared light, the Arduino adds one to the amount of bill dispensed.

An LCD display is shown to the left with “price: $” printed on it, a keypad installed in a pink box is shown to the right with the numbers 0 to 9, a star key, and a pound sign arranged onto it in a 3 by 4 matrix. When a key is pressed on the keypad, the display shows the updated number and when done, the user presses the pound sign to enter the price needed to be dispensed. The star key is used to delete unwanted numbers.

A red push button is placed to the right of the money roller. When this button is pressed, both yellow motors attached to the rollers will begin driving (the top one drives faster than the bottom to ensure the fabric doesn’t create slack between the rollers) to reload the money.

A bill is shown being dispensed by the money roller module. The module is made with two tubes and a black fabric that wraps around both of them. Bills are stored between the rolled-up fabric and motors are driven to dispense and reload the bills.

A video that demonstrates the entire process of dispensing money from the view of the user.

A video that demonstrates the entire process of dispensing money from the view of the shop clerks.

Narrative Sketch:

When Allie goes out shopping and is about to pay for her groceries, she would turn to the Money Dispenser sitting on her desk on top of the wheelchair. She types in the price of the item using the keypad and the motors run and dispense the correct number of 10- and 20-dollar bills. The display screen reads the change Allie would get back. When Allie gets home, she and her helper would unlock the cabinet in the front, and then press the red button which says reload 10s and reload the 20s to reload the machine for purchases in the future.

How we got here:

Prototype:

This prototype was designed to help answer the question: how to most effectively dispense only one bill at a time?

Our first prototype was based on how a printer dispenses paper and utilized multiple roller mechanisms that would dispense a bill from a stack. Basically a spring would push up a stack of bills to a certain height and then a rolling cylinder with grippy material would push the bill forward. We then moved on to our second prototype which is the design that we ended up using with two rollers and fabric pulled between them. This design is essentially a piece of fabric taped to two roller and as its winded, bills are inserted between the folds of the fabric. The fabric is then unwound to dispense the bills.

Our original bill-dispensing prototype

The first prototype required us to make a spring which we did by wrapping wire around a clamp.

First rolling mechanism with only two motors and one set of rollers.

Our first prototype of the bill dispensing mechanism was not able to consistently roll out a single bill at a time.

Prototype of the new bill dispensing mechanism without fabric attached

Testing the new bill-dispensing mechanism

Assembling our prototype for our final design

Most of our prototyping process was spent figuring out if it was possible for us to create a money dispenser, specifically one that would only dispense one bill at a time. We first modeled our mechanism after a printer that utilized a set of rollers that would only allow one bill to pass through at a time. However, we found that this mechanism required high precision tolerance between the rollers which was difficult for us to consistently achieve. We then switched to our final mechanism which instead of storing the bills in a stack where they could stick together, layered the bills separately inside a roll of fabric.

We received feedback that our original mechanism was flawed because we did not have a way to detect if the correct number of bills had been dispensed, which we solved by integrating IR sensors into the design. However, we weren’t able to react to the feedback to make the mechanism smaller due to how much testing that would require and the time limit we were under. The biggest (good) surprise we encountered was how smoothly our second mechanism worked after having so much difficulty with tweaking the first one.

Process:

Constructing the bill rollers using laser-cut and 3D-printed parts

Fully constructed bill rollers using standardized 3D-printed and laser-cut parts.

Figuring out the row and column pin-mapping for the keypad.

Wiring the IR receiver and transducer on a breadboard for initial readings

Integrating multiple pieces of hardware into a circuit with the Arduino Mega.

Constructing the back panel of the box

The biggest change in our design moving forward from our prototype presentation was switching our bill dispensing mechanism to a simpler two-roller setup. We found a mechanism online that used two rollers driven by motors to reload and dispense bills from a spool of fabric. This mechanism gave us more control over how many bills were released at once. It was also more appropriate for Allie’s needs since she told us that she gets about 60 dollars each week in the form of 3 twenty-dollar bills, so she didn’t necessarily need a device that could fit a large stack of bills. 

We also had to make a decision about how our device could track the number of dispensed bills from each compartment. Initially, we planned to time how long it would take to dispense a single bill, but that would require the bills to be evenly spaced out when loaded into the rollers. Instead, we chose to use an IR transmitter and receiver that detects a decrease in the IR signal as a bill exits the money rollers. Similarly, we had to find a method to determine when a given compartment was out of bills. At first we planned to use an RGB color sensor to detect a color change when a roller reached a white piece of tape at the end of the fabric. However, we ended up using an IR sensor so that we would only have to work with one signal value rather than the three values that an RGB sensor outputs. 

Our biggest breakthrough was figuring out how to prevent fabric slack from bunching up between the two rollers when running the motors. We found the best way to maintain good tension in the fabric roll was to drive the bottom roller faster than the top roller while dispensing and drive the top roller faster while reloading. This allowed us to smoothly and consistently dispense/reload bills. 

One mistake we made was waiting too long to start arranging our pieces of hardware into the outer shell of the box. We spent a lot of time fine-tuning the IR sensor thresholds outside the final cash box. This took up a lot of time, and we had to redo the IR signal thresholds in our code anyway because the box provided a much darker environment. If we had put our entire device into the box earlier on, we would have still been able to fine-tune the performance of our hardware and would have had more time for wire management inside the final form.

Gantt Chart Schedule:

Gantt chart timeline from prototype critique until final presentation.

Our team followed the sequence of tasks on the Gantt chart pretty closely. However, we underestimated how long it would take us to complete certain tasks. There ended up being a lot more hardware than initially expected, so the process of wiring, writing code, and introducing each component into the circuit took longer than expected. Rather than finishing the “implementing circuit” phase by 4/24, we didn’t finish integrating all the electronics and the roller mechanisms until 4/29. This only gave us 3 days to put all of our hardware into the outer box.

Conclusions and Lessons Learned:

This project has been a difficult but rewarding experience. We have run into a lot of problems during every stage of the process, including revamping the entire mechanism after the prototype review, but we were able to put together the mechanism in the end. The box itself during the final review still had a lot of room for improvement. As pointed out by comments such as “the fabrication is flimsy with a less-than-ideal UI”, “a bit big”, and “[Allie] can’t see what comes out”, the fabrication was a last-minute process and had flaws such as its size and fragility. We also planned on making a locked cabinet in the front but couldn’t get around to installing it. Additionally, the box orients itself away from Allie which prevents her from seeing the money being dispensed. A great solution that our critic brought up was adding a “swivel” to the bottom of the device so that it can “turn towards Allie” first, then when Allie makes sure it is the right amount, she can turn the machine towards the shop staff. Moreover, the problem of size was also discussed heavily in the critique. One critic suggested using “bill recognition” and some forms of servo motor that directs the money out or back into the loop so that we can use one module for all kinds of bills instead of two modules for two types of bills. This will not only increase the utility of the machine but also downsize it to a reasonable and usable form. Another critic pointed out is the messy “inner components” that should be organized which might reduce the size of the box further. However, despite the shortcomings, our project still got some praise for tackling such a difficult problem, dealing with single sheets of paper. Some critics said that the “motor system works well” and contains “clever paper handling” which we are happy to see.

Working with Allie has been a phenomenal experience as she was extremely helpful during our creation process. Allie shared us some hobbies with us and even some of the posters she made in her free time. We were able to learn to incorporate ergonomic designs into the project that caters to Allie’s specific needs, and through that, we learned more about disability beyond the project as we realized that disability is not just simply a medical definition but also a social problem. Some items and facilities are simply not designed for those with different needs and people are handicapped by the products and that is an external problem instead of an internal one. In the future, we will ask more questions working with people with disabilities and have more meetings with them to get a better sense of the user’s experience.

Working on Zoom certainly brought up some difficulties during the interview as Allie had some issues with her microphone, but overall, we as a team were able to get together in person for the most part and work in the Physical Computing lab together without any issues.

Photo with Allie at the final critique.

Technical Details:

Cash dispenser electrical schematic

Cash dispenser block diagram

Code:

/* Money Box Code - Andy Jiang, Cassie Rausch, Lynn Rushkin
 *  
 * How it works:
 *  Use the keypad to enter the price you would like to pay:
      '#' = enter (E)
      '*' = delete (D)
      type in your price as dollars and cents (i.e. 1600 --> $16.00, 4567 --> $45.67)
      if you mess up a digit then click '*' to delete the last digit
      press # when you have your price entered
   The code calculates how many 10 and 20 dollar bills need to be dispensed. As the device dispenses the bills, "dispensing..." will show up on LCD display. The IR transducer/receiver pair on the $10 and $20 compartments will detect how many bills have been dispensed and once the correct amount has been paid the motors of the compartments will stop running. The change amount is calculated based on how much was paid and the entered price. This amount will pop up on the LCD display. If one bill compartment is empty(detected by IR sensor)then the number of bills needed will be recalculated and dispensed from the compartment that still has bills. 

   Sources:
   1. Keypad code from Terrance Ou's "Math Buddy" code (https://courses.ideate.cmu.edu/60-223/s2022/work/math-buddy/)
   2. LCD display code from https://courses.ideate.cmu.edu/60-223/s2022/tutorials/I2C-lcd

  //----------------pin mapping ----------------------
  Pin mapping:
    Arduino Mega pin | type   | description
    -----------------|--------|-------------
    A1                 input     IR receiver 
                                 (detect dispensed $20 bills)
    A2                 input     IR receiver 
                                 (detect dispensed $10 bills)
    A4                 input     IR sensor 
                                 (detects empty $10 bill compartment) 
    A5                 input     IR sensor 
                                 (detects empty $20 bill compartment) 
    2                  output    toy gear box motor 1 
                                 (top roller of $10 bill compartment)
    3                  output    toy gear box motor 1 
                                 (top roller of $10 bill compartment) 
    5                  output    toy gear box motor 2 
                                 (bottom roller of $10 bill compartment)
    6                  output    toy gear box motor 2 
                                 (bottom roller of $10 bill compartment)
    9                  output    toy gear box motor 3 
                                 (top roller of $20 bill compartment)
    10                 output    toy gear box motor 3 
                                 (top roller of $20 bill compartment)
    11                 output    toy gear box motor 4 
                                 (bottom roller of $20 bill compartment)
    12                 output    toy gear box motor 4 
                                 (bottom roller of $20 bill compartment)
    25                 input     button 1 (for reloading $20 bills) 
    27                 input     button 2(for reloading $10 bills)
    39                 input     keypad (COL 2)
    41                 input     keypad (ROW 4)
    43                 input     keypad (COL 3)
    45                 input     keypad (ROW 1)
    47                 input     keypad (COL 1)
    49                 input     keypad (ROW 2)
    51                 input     keypad (ROW 3)
    SDA                          LCD display
    SCL                          LCD diaplay

   Note: toy gear box motors are connected to Arduino Mega through a dual motor driver
*/

//--------------initialize LCD----------------------
#include <Wire.h>
#include <LiquidCrystal_I2C.h>
LiquidCrystal_I2C screen(0x27, 16, 2);

//---------------------initialize keypad---------------------
#include <Keypad.h>
const byte ROWS = 4; // four rows
const byte COLS = 3; // three columns

char keys[ROWS][COLS] = {
  {'1', '2', '3'},
  {'4', '5', '6'},
  {'7', '8', '9'},
  {'D', '0', 'E'}
};


byte rowPins[ROWS] = {45, 49, 51, 41}; // connect to the row pinouts of the keypad
byte colPins[COLS] = {47, 39, 43};    // connect to the column pinouts of the keypad


Keypad keypad = Keypad( makeKeymap(keys), rowPins, colPins, ROWS, COLS );

//----------- initialize motors ----------------------------------
//pins for 20s top
int motorControlPin1 = 12;
int motorControlPin2 = 11;

//pins for 20s bottom
int motorControlPin3 = 2;
int motorControlPin4 = 3;

//pins for 10s top
int motorControlPin5 = 5;
int motorControlPin6 = 6;

//pins for 10s bottom
int motorControlPin7 = 9;
int motorControlPin8 = 10;

//---------------initialize IR receivers for bill detection ------------------
const int IRPin_20s = A1;
const int IRPin_10s = A2;
unsigned long LastIRTime;

int IRVal_20s; //reading of the sensor (0 to 1023)
bool currentIR_20s = 0; //is the IR sensor covered currently? (1 = yes, 0 = no)
bool prevIR_20s = 0; //was the IR sensor covered previously? (1 = yes, 0 = no)

int IRVal_10s;
bool currentIR_10s = 0;
bool prevIR_10s = 0;

int dispensedTwenties = 0; //counts how many bills have been dispensed from 20s compartment
int dispensedTens = 0;

//-------------initialize IR sensor variables for empty compartment detection -------
const int tenIR_empty = A4;
const int twenIR_empty = A5;
int tenIR, twenIR;
bool EmptyModule_20s = 0; //is the $20 compartment empty? (1 = yes, 0 = no)
bool EmptyModule_10s = 0;

// -----------initialize reload buttons ------------------------
const int tens_button = 27;
const int twens_button = 25;

int twens_buttonVal; //on or off values of the button reading
int tens_buttonVal;

//----------- initialize bill calculating variables --------------
char key; //keypad reading
String price = "";
float finalPrice = 0;
int billsTwenties = 0; //number of $20 bills we need to dispense
int billsTens = 0; //number of $10 bills we need to dispense
float change = 0;

// -----------calibrate these values -----------------------------

//motor speed dispense
const int twenTopSpeed_dispense = 100;
const int twenBotSpeed_dispense = 140;

const int tenTopSpeed_dispense = 100;
const int tenBotSpeed_dispense = 140;

//motor speed reloading
const int twenTopSpeed_reload = 150;
const int twenBotSpeed_reload = 80;

const int tenTopSpeed_reload = 150;
const int tenBotSpeed_reload = 60;

/*IR bill dispense threshold - any IR reading below these values means
a bill is passing */
const int twenBills_threshold = 50;
const int tenBills_threshold = 60;

/*IR empty module threshold - any IR reading above these values means 
the bill compartment has been emptied.*/
const int empty_threshold20s = 200;
const int empty_threshold10s = 180;

//------------setup and main loop---------------------------------

void setup() {
  Serial.begin(9600);

  // setup LCD
  screen.init();
  screen.backlight();
  screen.home();

  pinMode(motorControlPin1, OUTPUT);
  pinMode(motorControlPin2, OUTPUT);

  pinMode(motorControlPin3, OUTPUT);
  pinMode(motorControlPin4, OUTPUT);

  pinMode(motorControlPin5, OUTPUT);
  pinMode(motorControlPin6, OUTPUT);

  pinMode(motorControlPin7, OUTPUT);
  pinMode(motorControlPin8, OUTPUT);

  pinMode(tenIR_empty, INPUT);
  pinMode(twenIR_empty, INPUT);
  pinMode(IRPin_20s, INPUT);
  pinMode(IRPin_10s, INPUT);
  pinMode(twens_button, INPUT);
  pinMode(tens_button, INPUT);
}

void loop() {
  enteredPrice();
  if (key == 'E') {
    delay(1000);
    calculateBills();
    dispenseBills();
    stop10s_motor();
    stop20s_motor();
    delay(1000);
    calculateChange();
    resetVariables();
  }
 
  checkReload();
}

//------------functions -------------------------------------
void clearLastInput() {
  int lastIdx = price.length() - 1;
  screen.setCursor(lastIdx + 8, 0);
  screen.print(" ");
}

void enteredPrice() { //displays the price typed into the keypad on LCD
  screen.home();
  screen.print("price: $");
  key = keypad.getKey();
  if (key != NO_KEY
      && key != 'D' && key != 'E'
      && price.length() < 20) {
    price += key;
    screen.setCursor(8, 0);
    screen.print(price);
  }
  if (key == 'D') {
    clearLastInput();
    price.remove(price.length() - 1); //removes the last digit from the price string
  }

  else if (key == 'E') {
    finalPrice = price.toFloat() / 100;
    screen.setCursor(8, 0);
    screen.print(finalPrice);
    price = "";
  }
}

void calculateBills() { //calculates the number of 10 and 20 dollar bills to ideally dispense
  billsTwenties = finalPrice / 20; //integer divide to get how many twenties we need to dispense
  float remainder = finalPrice - 20 * billsTwenties;
  if (remainder > 10) {
    billsTwenties += 1;
  }
  else if (remainder > 0 ) {
    billsTens = 1;
  }

  /*Serial.print("bills twenties:");
  Serial.println(billsTwenties);
  Serial.print("bills Tens:");
  Serial.println(billsTens);*/
}

void run20s_motor() { //drives the top and bottom motors of the $20 compartment to dispense
  //do an initial acceleration (at highest speed) to overcome internal gear resistance, 
  //and then drive the motors at a reasonable speed
  
  if (digitalRead(motorControlPin1) == 0 && digitalRead(motorControlPin4) == 0) {
    analogWrite(motorControlPin1, 255);
    analogWrite(motorControlPin4, 255);
    delay(300);
  }

  analogWrite(motorControlPin1, twenTopSpeed_dispense);
  analogWrite(motorControlPin2, 0);

  analogWrite(motorControlPin3, 0);
  analogWrite(motorControlPin4, twenBotSpeed_dispense);

}
void stop20s_motor() {
  analogWrite(motorControlPin1, 0);
  analogWrite(motorControlPin2, 0);

  analogWrite(motorControlPin3, 0);
  analogWrite(motorControlPin4, 0);
}
void run10s_motor() {
  if (digitalRead(motorControlPin6) == 0 && digitalRead(motorControlPin8) == 0) {
    analogWrite(motorControlPin6, 255);
    analogWrite(motorControlPin8, 255);
    delay(300);
  }
  analogWrite(motorControlPin5, 0);
  analogWrite(motorControlPin6, tenTopSpeed_dispense);

  analogWrite(motorControlPin7, 0);
  analogWrite(motorControlPin8, tenBotSpeed_dispense);

}

void stop10s_motor() {
  analogWrite(motorControlPin5, 0);
  analogWrite(motorControlPin6, 0);

  analogWrite(motorControlPin7, 0);
  analogWrite(motorControlPin8, 0);
}

void checkIR() { //updates boolean values of whether a bill is passing
  //check IR readings in $20 compartment
  IRVal_20s = analogRead(IRPin_20s);
  if (IRVal_20s < twenBills_threshold) {
    currentIR_20s = 1;
  }
  else {
    currentIR_20s = 0;
  }

  //check IR in $10 compartment
  IRVal_10s = analogRead(IRPin_10s);
  if (IRVal_10s < tenBills_threshold) {
    currentIR_10s = 1;
  }
  else {
    currentIR_10s = 0;
  }
}

void count_billsdispensed() { /*uses the change in currentIR_10s value from 1 --> 0 
  to determine that a bill has passed*/
  if (millis() - LastIRTime >= 400) {
    LastIRTime = millis();
    checkIR();
    if (currentIR_20s == 0 && prevIR_20s == 1) {
      dispensedTwenties += 1;
    }
    prevIR_20s = currentIR_20s;
    if (currentIR_10s == 0 && prevIR_10s == 1) {
      dispensedTens += 1;
    }
    prevIR_10s = currentIR_10s;
    /*
      Serial.print("dispensed tens: ");
      Serial.println(dispensedTens);
    */
  }
}

void moduleStatus() { //check if either bill compartment has been emptied
  tenIR = analogRead(tenIR_empty);
  twenIR = analogRead(twenIR_empty);
  //Serial.println(tenIR); // push the most recent value to the computer
  if (tenIR >= empty_threshold10s) {
    EmptyModule_10s = 1;
  }
  if (twenIR >= empty_threshold20s) {
    EmptyModule_20s = 1;
  }
}

void dispenseBills() {
  screen.clear();
  screen.home();
  screen.print("dispensing...");
  
  //the following while loop will continue to run as long as there are bills needed to be dispensed to pay the entered price and at least one bill compartment hasn't been emptied
  while ((billsTwenties > dispensedTwenties or billsTens > dispensedTens) && (EmptyModule_20s == 0 or EmptyModule_10s == 0)) {
    if (billsTwenties > dispensedTwenties and EmptyModule_20s == 0) {
      run20s_motor();
    }
    if (billsTwenties > dispensedTwenties and EmptyModule_20s == 1) {
      stop20s_motor();
      billsTens += 2 * (billsTwenties - dispensedTwenties);
      billsTwenties = dispensedTwenties;
    }
    if (billsTwenties == dispensedTwenties) {
      stop20s_motor();
    }
    if (billsTens > dispensedTens and EmptyModule_10s == 0) {
      run10s_motor();
    }
    if (billsTens > dispensedTens and EmptyModule_10s == 1) {
      stop10s_motor();
      billsTwenties += ((billsTens - dispensedTens) / 2) + 1 ;
      billsTens = dispensedTens;
    }
    if (billsTens == dispensedTens) {
      stop10s_motor();
    }
    //check whether the 10s or 20s modules are empty
    moduleStatus();

    //check how many 10s and 20s have been dispensed
    count_billsdispensed();
  }
}

void calculateChange() {
  change = (dispensedTwenties * 20 + dispensedTens * 10) - finalPrice;
  screen.clear();
  screen.setCursor(0, 0);
  screen.print("change: $");
  screen.setCursor(9, 0);
  screen.print(change);
  delay(3000);
  screen.clear();
}

void checkReload() { //checks if buttons are being pressed. If so, then motors of the bill compartment that correspond to the button will be driven in the opposite direction to reload money into the rollers. 
  twens_buttonVal = digitalRead(twens_button);
  tens_buttonVal = digitalRead(tens_button);
  if (twens_buttonVal == 1) {
    if (digitalRead(motorControlPin2) == 0 && digitalRead(motorControlPin3) == 0) {
      analogWrite(motorControlPin2, 255);
      analogWrite(motorControlPin3, 255);
      delay(300);
    }

    analogWrite(motorControlPin1, 0);
    analogWrite(motorControlPin2, twenTopSpeed_reload);

    analogWrite(motorControlPin3, twenBotSpeed_reload);
    analogWrite(motorControlPin4, 0);

  }

  if (twens_buttonVal == 0) {
    stop20s_motor();
  }

  if (tens_buttonVal == 1) {
    //10
    if (digitalRead(motorControlPin5) == 0 && digitalRead(motorControlPin7) == 0) {
      analogWrite(motorControlPin5, 255);
      analogWrite(motorControlPin7, 200);
      delay(300);
    }

    analogWrite(motorControlPin5, tenTopSpeed_reload);
    analogWrite(motorControlPin6, 0);

    analogWrite(motorControlPin7, tenBotSpeed_reload);
    analogWrite(motorControlPin8, 0);
  }
  
  if (tens_buttonVal == 0) {
    stop10s_motor();
  }
}

void resetVariables() {
  key;
  price = "";
  finalPrice = 0;
  billsTwenties = 0;
  billsTens = 0;
  change = 0;
  dispensedTwenties = 0; //counts how many bills have been dispensed from 20s compartment
  dispensedTens = 0;
  EmptyModule_20s = 0;
  EmptyModule_10s = 0;
}

Design Files

Money box case zip

Money dispenser zip

Keypad box

The Inspiration Capture Device (ICD) is the product of weeks of collaboration with our team’s client, Mary. After meeting with Mary multiple times in person and through the phone, we determined that Mary, as a writer, values the inspiration that can come at a moment’s notice highly. Thus, we wanted to make her an assistive device that allowed her to quickly and easily capture whatever kind of inspiration reaches her in her daily life so that it could help her writing process.  For some more background information on our project, please refer to this post: https://courses.ideate.cmu.edu/60-223/s2022/work/team-callisto-interview-documentations/

What we built

The ICD captures ‘inspiration’ in two different ways- through a picture taken on a small camera or through a voice memo recorded on an audio recorder. The ICD is meant to do this while being as accessible as possible for Mary; the device is split into two parts (a remote controller and a necklace) to make it easy for her to interact with. At the mere press of a button, Mary can take a picture of whatever she is currently facing or turn on a voice recorder on her neck to quickly capture what’s on her mind for future use.

Camera component of the ICD’s remote controller

Inside the remote you’ll notice various wires as well as an Arduino Uno and a transceiver on the left near the camera

Top view of the remote controller, featuring two buttons with their LED lights meant to indicate current remote activity

The necklace part of the ICD featuring a green LED

The above video is a compilation of clips relating to the ICD’s capabilities.

Image of Juan’s hand that was captured in the ICD demo video.

Narrative

While Mary is out shopping, she notices that a beautiful unfamiliar kind of bird has perched itself on a table in front of her. She pushes the ICD’s red button to take a picture of the bird so she could find out what type of bird it is later. After doing so, she decides to write something about the bird.

Later in the week, she suddenly gets a fantastic idea for a short story  while she’s out and about in Pittsburgh. To ensure she doesn’t forget this idea, she presses the ICD’s blue button and says her idea out loud. She then presses the button again to stop and save the recording. Hours later, Mary uploads the file on her computer, listens to her voice memo, and starts getting to work on an amazing story.

Prototype and Process

Our prototype was broken into four parts so that we could test each component before combining them.

The four components included the remote control, the microphone, the camera, and the necklace. The remote control included large buttons connected to LEDs and wireless transmitters, the microphone and the camera were each attached to an SD card, and the necklace included a casing for internal components and a chain.

The remote control component of the ICD prototype

The microphone component of the ICD prototype

The camera component of the ICD prototype

The necklace component of the ICD prototype

The above video showcases the ICD’s simple user interface.

Prototyping

The initial sketches for the necklace part of the ICD

Through our interview with Mary, we found that she is creative and a writer but cannot write down her thoughts quickly, so we created a solution that would allow her to record herself the moment she receives inspiration.

The initial sketch for the remote control part of the ICD

After talking with Mary about our initial sketches and solutions, she suggested that we also include a camera capability because she needs the help of someone else to use her phone camera. We also decided to create a separate remote that housed the camera but also triggered the microphone in the necklace wirelessly to make it more accessible for Mary.

The initial sketch for the attachment of remote

We were trying to figure out the best way to attach the remote to the wheelchair so that it would be easy for Mary to press to record her life. We found that it might be best for remote attachment works like a watch to fix on the wheelchair controller.

Started a simple setup to work on the microphone component using a microsd card reader module and a microphone module

Upon recording the audio noticed that there was a lot of noise. In order to investigate where the noise was coming from we hooked the output of the microphone to an oscilloscope. With some testing we realized that the noise was coming from writing to the SD card reader.

During the prototype critique, we asked Mary to interact with the microphone set up, focusing on how she would insert and remove the microsd card from the reader. We found that reader module was not optimized for Mary’s hands as the sd card was too small and the push mechanism to remove was difficult. And so, we invested in a larger SD card reader.

Unfortunately, the noise still persisted. We switched the arduino uno for an arduino pro micro to see if the board made any difference. Our final design would also use a pro micro as it would allow the microphone to fit in the necklace holder.

The above change did not fix the noise either. As a last resort we scrapped the SD card readers and invested in a module(vs1053)  that had the optimizations required to cleanly record audio. Given more time we would have fully recorded audio with little noise, but instead the audio files were written to the SD card without any audio.

There were also some issues with other components; a moment of frustration regarding the transceivers in our project should be mentioned: Juan spent an unideal amount of time attempting to debug issues with the wireless transceivers. Unsure if the issue was hardware, software, or wiring-related, hours were lost on his part trying to fix the transceiver problem and could’ve gone towards other aspects of the project. This narrow focus on solving the problem in retrospect was a mistake- the time spent trying to resolve the issue (which ended up being due to a broken transceiver) could have instead been spent assisting Sumayya with the voice recorder component of the project. By the time the solution to the transceiver problem was discovered, it was simply too late to assist with other aspects of the project. This served as a learning experience regarding debugging physical computing components as well as knowing when to cut your losses. In addition, we also had some issues with the ICD’s physical case.

The failed versions of the cases

To make sure the two cases are easy for Mary to use, we have to make them both as small and light as possible. Therefore, every electronic component should be located at its specific place. We did a lot of tests to decrease the size.

Prototyping Process Findings

Though we each worked on separate components of the ICD, the prototyping process helped teach us a variety of lessons not only about our individual components, but also about working on a physical computing project in general. From questions regarding how our overall project would be powered to what final form it would take, there were several moments throughout the prototyping process where we had to answer questions as a team rather than individually. This was somewhat surprising-even though we were all working on vastly different pieces of technology, the choices we each made on our parts had effects on the overall team project that were sometimes subtle and other times drastic. It was partly due to this that our final version was incomplete- we didn’t truly begin accommodating each other’s work until the final stages of the project. By that point, it was too late to make some much needed adjustments, such as making the ICD’s case less sharp and improving the ICD’s attachment to a wheelchair which were recommended to us from our critiques. We did not intentionally ignore this feedback, but unfortunately we ran out of time to address most of it.

Gantt Tracking

Our team’s Gantt chart- visualizing our originally planned project schedule

Overall, we were able to roughly follow our Gantt chart to about 75% accuracy. Divergences from our schedule were mainly due to longer than expected debugging times, slow-arriving components that were stuck in transit for longer than preferred, as well as other miscellaneous issues such as group members falling ill during some days.

Conclusions and Lessons Learned

Feedback

Some feedback that stood out to us were the following:

• “Develop ergonomics, far reach in objectives file transferring”
• “There are several steps involved in retrieving files; but, those could be incorporated into an end-of-day routine. The device is a bit large and the sharp corners could be uncomfortable.”
• “They really should have considered how their client would use this and added some personality of their client into it.”
• “A good idea. Make attachment to chair more stable.”

The reason why these pieces of feedback stood out to us was because they reinforced ongoing discussions we were already having regarding the direction of our project. Adding “personality” and making file transferring simpler, for instance, were goals we wanted to achieve but didn’t really figure out how to do so. This was partly due to delays in our planned schedule, and ultimately by the time we shifted our focus to the form of our project there was too little time to implement some of the recommended changes. That being said, all of the feedback received was insightful and extremely helpful for possible future iterations of our project.

Thoughts on Remote Work

In general, it was smooth to work remotely. At the first beginning, we lost contact with Mary but we got into a collaboration with her successfully soon. Sometimes we did different parts of the project individually but we still cared about the work from each other. Everyone put a lot of effort into this project and often continued working after class alone. If we had another chance to work together remotely, it might be a good way for better collaboration by updating the process once after work.

Takeaways from working with Mary

This journey looked like we were doing a device to help Mary, but literally it was Mary who inspired us a lot. She never gave up her love towards life though her mobility was limited. She had her own hobbies and work, her life was so adorable and she got a lot of sense of achievement during her tutorial work. “I really enjoy my life.” Mary said in her first interview with us. At that time we four were all touched by her strong positive vibe.

Concluding Thoughts

Even though our final prototype was somewhat distant from our original vision, the process we all went through while working on the prototype was very educational and insightful. We each learned different things about not just our components, but about the process of prototyping a physical computing project as well as working as a team in general.

Given a second go at the project knowing what we know now, our biggest changes would have to do with how we worked on the project in general. Our decision to split the ICD into four separate parts, one for each member, seemed like an obvious one at first but in retrospect proved to be a hindrance in the final stages of our prototyping process. Since we were all the resident “expert” on the part we were working on within the group, having a group member absent severely hampered our ability to move the project forward. This hurt us a lot after carnival, where some members of the group got sick and as a result had to miss some work sessions- negatively impacting the whole group’s progress.

A possible solution would be to assign two members to each component. This would greatly help with the aforementioned absence problem, and it would have also had positive effects on each component itself since there would be two minds tasked to making progress on each part instead of just one. Additionally, we would also resolve certain prototype issues, like choosing a battery and final form, as soon as possible instead of saving that for the very end given the importance of accessibility for our client.

Some other takeaways include establishing a debugging process for components early on, not underestimating the soldering process, properly gauging capabilities/knowledge early on in the process, and removing/combining features earlier! These takeaways were largely individual- Juan’s takeaway (the debugging process) was the result of too many hours spent trying to debug what ended up being broken components. Sumayya’s, on the other hand, centered on her experience tackling the complicated voice recording system. Instead of pivoting to a different direction earlier, her various attempts to get the component working backfired as other components that were behind in terms of progress would have made major use of an extra mind. Yongwen and Tristan also had their fair share of takeaways stemming from their assigned components. Overall, however, we can simply say that each one of us learned something valuable about our very own approaches to different problems.

Technical details

Diagrams

Remote Controller Diagram

Necklace Diagram

Code

// Thanks to ArduCAM Mini demo (C)2018 by Lee, found at http://www.ArduCAM.com
/*Image Capture Device - Remote/Camera Code,
Juan Meza, Tristan Hineman
This code operates the remote's various functionalities, including
how it takes user input, how it sends data wirelessly, and how it takes 
and saves images taken from the device. 
*/

//SPI Mosi-11, Miso-12, Sck-13

#include <nRF24L01.h>
#include <RF24.h>
#include <EEPROM.h>

#include <Wire.h>
#include <ArduCAM.h>
#include <SPI.h>
#include <SD.h>
#include "memorysaver.h"

#define BUTTON 8
#define BUTTON_BLUE 2
#define LED_BLUE 6 //blue LED
#define LED_RED 5 //red LED
#define SD_CS 9
#define FRAMES_NUM    0x00

const byte address[6] = "42069";
bool isButtonPressed = false;
unsigned long buttonTime;
int ButtonPresses = 0;

// set pin 7 as the slave select for the digital pot:
const int CS = 7;

RF24 radio(3, 4); // CE, CSN

//camera
//This demo can only work on OV5640_MINI_5MP_PLUS or OV5642_MINI_5MP_PLUS platform.
#if !(defined (OV2640_MINI_2MP_PLUS))
#error Please select the hardware platform and camera module in the ../libraries/ArduCAM/memorysaver.h file
#endif


bool is_header = false;
int total_time = 0;

#if defined (OV2640_MINI_2MP_PLUS)
ArduCAM myCAM( OV2640, CS );
#endif

uint8_t read_fifo_burst(ArduCAM myCAM);

void setup() {

  pinMode(BUTTON, INPUT_PULLUP);
  pinMode(BUTTON_BLUE, INPUT_PULLUP);
  pinMode(LED_BLUE, OUTPUT);
  pinMode(LED_RED, OUTPUT);

  buttonTime = millis();

  radio.begin();
  radio.openWritingPipe(address);
  radio.setPALevel(RF24_PA_MIN);
  radio.stopListening();

  // camera
  uint8_t vid, pid;
  uint8_t temp;
  
#if defined(__SAM3X8E__)
  Wire1.begin();
#else
  Wire.begin();
#endif

  Serial.begin(115200);
  
  Serial.println(F("ArduCAM Start!"));
  // set the CS as an output:
  pinMode(CS, OUTPUT);
  digitalWrite(CS, HIGH);
  // initialize SPI:
  SPI.begin();
  //Reset the CPLD
  myCAM.write_reg(0x07, 0x80);
  delay(100);
  myCAM.write_reg(0x07, 0x00);
  delay(100);
   
   
  while (1) {
    //Check if the ArduCAM SPI bus is OK
    myCAM.write_reg(ARDUCHIP_TEST1, 0x55);
    temp = myCAM.read_reg(ARDUCHIP_TEST1);
    if (temp != 0x55)
    {
      Serial.println(F("SPI interface Error!"));
      delay(1000); continue;
    } else {
      Serial.println(F("SPI interface OK.")); break;
    }
  }


#if defined (OV2640_MINI_2MP_PLUS)
  while (1) {
    //Check if the camera module type is OV2640
    myCAM.wrSensorReg8_8(0xff, 0x01);
    myCAM.rdSensorReg8_8(OV2640_CHIPID_HIGH, &vid);
    myCAM.rdSensorReg8_8(OV2640_CHIPID_LOW, &pid);
    if ((vid != 0x26 ) && (( pid != 0x41 ) || ( pid != 0x42 ))) {
      Serial.println(F("ACK CMD Can't find OV2640 module!"));
      delay(1000); continue;
    }
    else {
      Serial.println(F("ACK CMD OV2640 detected.")); break;
    }
  }
  
#endif


  //Initialize SD Card
  while (!SD.begin(SD_CS))
  {
    Serial.println(F("SD Card Error!")); delay(1000);
  }
  Serial.println(F("SD Card detected."));
  //Change to JPEG capture mode and initialize the OV5640 module
  myCAM.set_format(JPEG);
  myCAM.InitCAM();
  myCAM.clear_fifo_flag();
  myCAM.write_reg(ARDUCHIP_FRAMES, FRAMES_NUM);
}


void loop() {

  myCAM.flush_fifo();
  myCAM.clear_fifo_flag();
#if defined (OV2640_MINI_2MP_PLUS)
  myCAM.OV2640_set_JPEG_size(OV2640_1600x1200);
#endif

  int buttonVal = digitalRead(BUTTON);
  //Serial.println(buttonVal);
  //as a reminder, when a button is being pressed buttonVal turns to zero

  int buttonBlueVal = digitalRead(BUTTON_BLUE);

  if (buttonVal == 0) {
    Serial.println("red is being pressed");
    digitalWrite(LED_RED, HIGH);

    myCAM.start_capture();
    Serial.println(F("start capture."));
    total_time = millis();
    while ( !myCAM.get_bit(ARDUCHIP_TRIG, CAP_DONE_MASK));
    
    Serial.println(F("CAM Capture Done."));
    total_time = millis() - total_time;
    Serial.print(F("capture total_time used (in miliseconds):"));
    Serial.println(total_time, DEC);
    total_time = millis();
    
    read_fifo_burst(myCAM);
    total_time = millis() - total_time;
    Serial.print(F("save capture total_time used (in miliseconds):"));
    Serial.println(total_time, DEC);
    //Clear the capture done flag
    myCAM.clear_fifo_flag();
    digitalWrite(LED_RED, LOW); //turn off to signify that image has been taken
    
  }

  if (buttonBlueVal == 0) {
    Serial.println("blue is being pressed");
    const char text[] = "ACTIVATE NECKLACE";
    radio.write(&text, sizeof(text));
    digitalWrite(LED_BLUE, HIGH);
    delay(500); //do some blinking first
    digitalWrite(LED_BLUE, LOW);
  }

  if ((buttonVal == 0 || buttonBlueVal == 0) && millis() - buttonTime >= 500) { //button is being pressed for reasonable time

    //Serial.println(millis()-buttonTime);
    ButtonPresses++;
    buttonTime = millis(); //reset buttonTime for next button press
    //Serial.println("Button is being pressed.");
    Serial.println("Button has been pressed this many times:");
    Serial.println(ButtonPresses);

  }

  else {
    //Serial.println("Button is NOT being pressed.");
  }

  if (ButtonPresses >= 1) {
    //digitalWrite(LED1,HIGH); //on, LOW for off

  }

  if (ButtonPresses == 2) {
    //digitalWrite(LED2,HIGH);

  }

  if (ButtonPresses > 2) { //more than 2,  reset!
    Serial.println("TIME TO RESET");

    digitalWrite(LED_BLUE, LOW);
    digitalWrite(LED_RED, LOW);

    delay(300); //do some blinking first

    digitalWrite(LED_BLUE, HIGH);
    digitalWrite(LED_RED, HIGH);

    delay(300);

    digitalWrite(LED_BLUE, LOW);
    digitalWrite(LED_RED, LOW);

    ButtonPresses = 0; //reset total button presses
  }
}

uint8_t read_fifo_burst(ArduCAM myCAM)
{
  uint8_t temp = 0, temp_last = 0;
  uint32_t length = 0;
  static int i = 0;
  static int k;
  char str[16];
  File outFile;
  byte buf[256];
  length = myCAM.read_fifo_length();
  Serial.print(F("The fifo length is :"));
  Serial.println(length, DEC);
  
  if (length >= MAX_FIFO_SIZE) //8M
  {
    Serial.println("Over size.");
    return 0;
  }
  if (length == 0 ) //0 kb
  {
    Serial.println(F("Size is 0."));
    return 0;
  }
  
  myCAM.CS_LOW();
  myCAM.set_fifo_burst();//Set fifo burst mode
  i = 0;
  while ( length-- )
  {
    temp_last = temp;
    temp =  SPI.transfer(0x00);
    //Read JPEG data from FIFO
    if ( (temp == 0xD9) && (temp_last == 0xFF) ) //If find the end ,break while,
    {
      buf[i++] = temp;  //save the last  0XD9
      //Write the remain bytes in the buffer
      myCAM.CS_HIGH();
      outFile.write(buf, i);
      //Close the file
      outFile.close();
      Serial.println(F("OK"));
      is_header = false;
      myCAM.CS_LOW();
      myCAM.set_fifo_burst();
      i = 0;
    }
    if (is_header == true)
    {
      //Write image data to buffer if not full
      if (i < 256)
        buf[i++] = temp;
      else
      {
        //Write 256 bytes image data to file
        myCAM.CS_HIGH();
        outFile.write(buf, 256);
        i = 0;
        buf[i++] = temp;
        myCAM.CS_LOW();
        myCAM.set_fifo_burst();
      }
    }
    
    else if ((temp == 0xD8) & (temp_last == 0xFF))
    {
      is_header = true;
      myCAM.CS_HIGH();
      //Create a avi file
      EEPROM.get(0, k);
      k = k + 1;
      EEPROM.update(0, k);
      
      Serial.print("Picture number=");
      Serial.println(k);
      itoa(k, str, 10);
      strcat(str, ".jpg");
      
      //Open the new file
      outFile = SD.open(str, O_WRITE | O_CREAT | O_TRUNC);
      
      if (! outFile)
      {
        Serial.println(F("File open failed"));
        while (1);
      }
      
      myCAM.CS_LOW();
      myCAM.set_fifo_burst();
      buf[i++] = temp_last;
      buf[i++] = temp;
    }
  }
  myCAM.CS_HIGH();
  return 1;
}

/*               
* Thanks to Dejan Nedelkovski, www.HowToMechatronics.com
/*Image Capture Device - "Necklace" Code,
Juan Meza 
This code operates our necklace's wireless capabilities.
It where the necklace should look for a signal and 
performs an action after finding said signal. 
*/


#include <SPI.h>
#include <nRF24L01.h>
#include <RF24.h>
#include <string.h>

#define LED_TEST 3 //green temp LED

const byte address[6] = "42069";
RF24 radio(7, 8); // CE, CSN

void setup() {

  pinMode(LED_TEST,OUTPUT);
  
  Serial.begin(115200); 
  radio.begin(); 
  radio.openReadingPipe(0, address); 
  radio.setPALevel(RF24_PA_MIN); 
  radio.startListening(); 
  
  Serial.println("Receiver is ready to receive...");
  digitalWrite(LED_TEST,HIGH);
  delay(300); 
  digitalWrite(LED_TEST,LOW); 
  
}

void loop() {
  
  if (radio.available()) {
    char text[32] = "";
    radio.read(&text, sizeof(text));
    Serial.println(text);

    String key = "ACTIVATE NECKLACE";

    if (String(text) == key){
      //blink LED! 
      digitalWrite(LED_TEST,LOW); 
      digitalWrite(LED_TEST,LOW); 
      delay(300); 
      digitalWrite(LED_TEST,HIGH);
      digitalWrite(LED_TEST,HIGH);
      delay(300);
    }
  }
  
}

Special thanks to Professor Zacharias and everyone else who helped us get so far!