Bill’s Cooking Buddy by Hue Values: Final Documentation

 

Introduction

In this class, we worked collaboratively with our client  Bill, who has achromatopsia and is sensitive to light. We aimed to create an assistive device to help him with his daily activities. After conducting several interviews with Bill, we learned that he loves cooking but has difficulty determining if the meat is cooked through due to his inability to see color changes. As a result, we decided to create an assistive cooking device to help him cook safely. We aim to provide Bill with a device that will enable him to cook confidently and safely.

What we Built

 

Our project is an assistive device designed to help our client Bill with his cooking activities. As mentioned, Bill has achromatopsia and is sensitive to light, which makes it difficult for him to determine if meat is cooked through due to his inability to see color changes. To address this challenge, our device has two systems. The first system is a temperature sensing system that allows Bill to input the desired cooking temperature using a keypad. The device will then display both the set temperature and the current temperature detected by the probe on two seperate screens. The second system is a timer that countdown the cooking time for Bill’s dish and alerts him when it’s completed. Together, these two systems help Bill cook safely and confidently without worrying about undercooked or overcooked meat.

 

 

“Story”

Bill wakes up early on a Sunday morning feeling energized and ready to cook up a storm in the kitchen. He decides to make a special breakfast of pancakes and bacon, and he can’t wait to use his new assistive cooking device. He pulls out his ingredients, preheats the stove, and starts cooking the bacon. With his device by his side, he inputs the desired temperature and sets the timer for the bacon. As the bacon cooks, Bill takes advantage of the time to prepare the pancake batter. He uses the probe to check the internal temperature, and the device displays the readings on the screen. The bacon is perfectly cooked, and Bill is thrilled with the results. Next, it’s time to cook the pancakes. Bill sets the temperature for the griddle and starts cooking. He sets the timer for each batch, knowing that he won’t have to worry about overcooking or undercooking them. The device beeps to signal when it’s time to flip the pancakes, and Bill easily flips them. Finally, it’s time to plate the food. Bill arranges the pancakes and bacon on the plate and pours a generous maple syrup. He sits down to enjoy his meal, feeling proud of himself for cooking such a delicious breakfast. Thanks to his new assistive cooking device, Bill can cook safely and confidently without worrying about undercooked or overcooked meat. He feels empowered and independent in the kitchen and can’t wait to try out new recipes with his device by his side.

How we got There

• Prototype

We started by prototyping the inputs and outputs of the meat-cooking assistive device. We were thinking about and exploring how different buttons, screens, LEDs, and temperature sensor work with one another and are housed and interacted with by Bill.

 

After the prototype critique, we asked more questions to Bill because we wanted to design a device that truly fits his preferences. Our questions for Bill after these initial prototypes revolved around:

1. Where would he like it? Storage? 
   • On the countertop
   • As a complete object (fully handheld) 
   • Something that fits in a drawer
   • Something that goes on a backsplash
   • Should it be wipable/ cleanable? 
2. What kind of inputs would he prefer? 
   • More touch screen style 
   • Different kinds of potentiometers 
   • Keypad  
   • The dirtiness of fingers may be a consideration
   • Does he want to set each degree of temperature or scroll through a menu? 
   • Does he want the additional feature of a cooking timer?
3. What kind of outputs to tell him that it is ready
   • Audio
   • Visual? Zach said he had a high screen
   • Light? How many bulbs are on kind of thing

Incorporating the feedback we received during the critiques, we made several changes to our design. We added a voice output feature for the current temperature and included a separate timer feature. We also made the box housing generally pretty dark for higher contrast. During the refining process, we asked Bill many questions about his cooking preferences and needs. We learned that while he does cook in a dark kitchen and likes sound output, he prefers to read numbers from a screen. He also mentioned that white text on a black background and a comfortable font size would work well for him. We found a 7-segment number display that would meet his needs and decided to make the housing and aesthetic of the box generally pretty dark to provide higher contrast.

Initially, we had considered including a progress bar or feedback feature to let Bill know how close his meat was to reaching the target temperature. However, Bill said it wouldn’t be something he would particularly need, so we decided to focus on providing him with the current and target temperatures on the two displays instead. Regarding input, we discussed the practicality of a keypad layout and decided to include a little button connecting to the speaker to voice the current temperature when pressed. We also included a separate timer feature with an hour and minute dials and a similar screen and button for voicing the feature.

Considering Bill wants to put this device on his shelf above the stove, we designed to used a power cable that allows Bill to plug it in and use the device easily. We also added an on and off switch just in case he doesn’t like to plug in and out every time but still can turn the device on and off.

Another feedback we took from the critique is that adding a hinge on the device to organize the probe is necessary. So we 3D modeled and printed a hinge that could exactly hold the probe and mounted it to the top of the device.

•   Process

Our Built process followed our initially outlaid schedule very closely, we did not experience a lot of delays or deviations from the initial design plan aside from the electronics taking 1 day extra to finish and debug.

3D Printed Hinge for the thermistor

Laser-cut the box with black acrylic. and soldered electronics placed side by side.

We have a double panel for the front to cover the ‘ABCD’ row of the keypad, which we will not use.

Soldered all electronics together to avoid unstable connections.

 

Glue the components on the box.

Overall the build process was what we expected. We stuck closely to our initial design and the electronics cooperated with the vision of our final product. There were a few strange library conflict with our text to speech library but that was debugged rather quickly. All of our box parts fit well and the foresight to use clearly labeled stranded wire made assembly very easy.

 

Conclusions and Lessons Learned

Starting with the final critique. We were overall pleased with the final presentation and critique of the project. There were some last minute software/ electric related bugs but other than that, our device performed as expected. 

 

The feedback that we received was roughly split into 2 categories: design and execution. The design related feedback was very insightful as at that point in time, we had been staring at this particular design for a couple weeks, having fresh perspectives was nice. Much of the feedback centered around the user friendliness of the design stating that :”Some of the usability was unclear, it could benefit from some labels to clarify what each numbers are corresponding to and a volume potentiometer.

“ and “Would be helpful if the buttons read out their purposes as well as the corresponding figures, as well as having the displays physically labeled”. The core of the feedback that complimented the design on its sleekness and minimal look also criticized the confusing nature of the placement of the buttons. It would probably be a good idea to label the buttons as it does not really detract from the overall look or function of the device. It was an issue of running out of time that we chose not to mess with the overall design too much. In addition, the displays being physically labeled would probably preclude the need for labeling the buttons. We also had a design comment about the placement of the thermometer over the speaker cutout. “Placement of thermometer rest in proximity to speaker holes presents moisture challenges and could be made more ergonomic.” This was very interesting as we had not considered the potential for the thermistor to become wet and leak into the speaker port. Given that both the port and the mount are easily moved. A future interaction would definitely address this. There was also a comment about adding some rubber feet to the device to prevent it from slipping. This would probably be a good idea on a commercial version but we designed the device to fit on a specific shelf where slipping when pushing a button would not cause any problems.

 

Finally on execution notes. A lot of notes were on the buggy timer and screen flashing which we were aware of potential fixes already. However, there were some software notes that we had not considered. One interesting note is that “Needs warning beeps at T–5°, solid alarm tone at T°, and a constant (but dismissable) BURN ALERT at something like T+10°.” While designing the functionality of the product, we only considered a warning when the temperature was met with the assumption that Bill would be able to head and see it. If it’s the case that he is in another room, we did not consider that possibility. A future iteration would include an approaching temperature chime as well as a burn warning that is sustained in order to address these cases. I think this was just a situation where we did not consider the possibility of him missing the complete chime. 

 

Overall the feedback was very insightful and having many experienced perspectives really helped us understand the advantages and disadvantages of the design choices that we made.

 

Working with Bill was an overall good experience. We think given that he was the coordinator and CLASS staff, our experience might be different from other groups working with CLASS members. With Bill specifically, something like not being able to see the doneness of meat was not a complication that we had ever considered with color blindness. Having him as a client made us aware of a lot of issues that partial vision deficiency could cause for a person even if it’s not as serious as full blindness. We are grateful for the opportunity to work with him. We also learned valuable lessons working with and interacting with the other clients in the project as well as the CLASS trip. Prior to this project, we did not have a lot of experience interacting with the disabled community and this project gave a lot of insight into their lives and struggles. 

 

Finally, some notes about this project. This project ran smoothly overall and the team worked well together. The division of tasks was more or less equal and people were able to bring their specialized skills to make the project what it was. Given the opportunity to do it again, our process would be very similar. We built it flex time in the event that something went wrong which helped us out a lot in the end. This final project was an excellent close to a very interesting and engaging class.

 

Technical Details

• Schematic and Block Diagram

 

• Codes

/*
Project Name: Bill's Cooking Buddy Helper
Project Members: Sapna Tayal, Angie (Chuyi) Wang, Harry Ren
Code Author: Harry Ren
- Assistive Meat Thermometer and Cooking timer
- Keypad input for target temperature
- All values displayed on 7 segments and can be read via speaker

Controller: Arduino Mega

Pin Mapping:
A0: Keypad pin 1
A1: Keypad pin 2
A2: Keypad pin 3
A3: Keypad pin 4
10: Keypad pin 5
11: Keypad pin 6
12: Keypad pin 7
13: Keypad pin 8

2: Current Temp Button
3: Target Temp Button
4: Timer Button

A8: Hour Potentiometer Pin
A9: Minute Potentiometer Pin

6: Main Audio Amp Pin
7: Inverted Audio Amp Pin

A10: Temp Probe Pin

Note: all 7 Segment pins connect to I2C Bus (master-slave) System
SDA: 7 Segment main SDA
SCL: 7 Segment main SCL

Portions of code are adapted from the following sources
- https://forum.arduino.cc/t/another-bbq-thermometer-wifi-enabled-with-push-notifications/225467
- https://forum.arduino.cc/t/countdown-timer-and-adafruit-4-digit-7-segment-display-w-i2c-backpack/364882/2
- https://www.instructables.com/Arduino-Text-to-Speech-Converter-Using-LM386-Talki/
relevant functions are credited
code by Harry Ren at Carnegie Mellon University (hjren@andrew.cmu.edu)
released by author to public domain, April 2023
*/


#include <Keypad.h>
#include <OneWire.h>
#include <DallasTemperature.h>
#include "Talkie.h"
#include "Vocab_US_Large.h"
#include <Wire.h>  // Enable this line if using Arduino Uno, Mega, etc.
#include <Adafruit_GFX.h>
#include "Adafruit_LEDBackpack.h"
#include <math.h>

//Lines 12-19 adapted from https://forum.arduino.cc/t/another-bbq-thermometer-wifi-enabled-with-push-notifications/225467/2
float air;          //  temperature
float meat;         //   Unused from sample code
float vin = 4.999;  //  DC Voltage as measured with DMM between +5V and GND
float r2 = 100000;  // Resistance in ohms of your fixed resistor

float A = 0.00004556837992;     // "A" Coeffecient in Steinhart-Hart Equation
float B = 0.0002634285613;      // "B"
float C = 0.00000005286310982;  // "C"

const int sw1 = 2;  //current temp button
const int sw2 = 3;  //tgt temp button
const int sw3 = 4;  //timer button
const int sw4 = 5; //io switch

Talkie voice;  //init speaker

// Define the pins for the 7-segment display: depreciated
const int digitPins[] = { 2, 3, 4, 5 };
const int segmentPins[] = { 6, 7, 8, 9, 10, 11, 12 };

// Define the keypad
const byte ROWS = 4;
const byte COLS = 4;
char keys[ROWS][COLS] = {
  { '1', '2', '3', 'A' },
  { '4', '5', '6', 'B' },
  { '7', '8', '9', 'C' },
  { '*', '0', '#', 'D' }
};
byte rowPins[ROWS] = { A0, A1, A2, A3 };  //keypad pins
byte colPins[COLS] = { 10, 11, 13, 12 };  //keypad pins
Keypad keypad = Keypad(makeKeymap(keys), rowPins, colPins, ROWS, COLS);


Adafruit_7segment mat = Adafruit_7segment();     //curr 7seg
Adafruit_7segment mat1 = Adafruit_7segment();    //tgt 7seg
Adafruit_7segment matrix = Adafruit_7segment();  //timer 7seg

int number = 0;  //curr tgt
int temp = 0;    //curr temp

int startMinute = 4;   //  Modify these defines to
int startSecond = 59;  // change the timer interval
int minutes = startMinute;
int seconds = startSecond;
bool notStarted = 0;  //timer started bool flag

unsigned long previousSecondMillis = 0UL;
long oneSecond = 1000UL;  // milliseconds per second

const int POT_PIN_H = A8;
const int POT_PIN_M = A9;
int digit_H = 0;
int digit_M = 0;


void setup() {

  Serial.begin(9600);

  //init button pins
  pinMode(sw1, INPUT);
  pinMode(sw2, INPUT);
  pinMode(sw3, INPUT);

  //init 7 segments
  mat.begin(0x70);
  mat1.begin(0x71);
  matrix.begin(0x72);
  mat.print(number);
  mat1.print(temp);

  matrix.writeDigitRaw(2,0x2);
  
  // Set the digit pins as outputs: depreciated
  for (int i = 0; i < 4; i++) {
    pinMode(digitPins[i], OUTPUT);
  }

  // Set the segment pins as outputs: depreciated
  for (int i = 0; i < 7; i++) {
    pinMode(segmentPins[i], OUTPUT);
  }

//init speaker code: depreciated
#if defined(TEENSYDUINO)
  pinMode(5, OUTPUT);
  digitalWrite(5, HIGH);  //Enable Amplified PROP shield
#endif
}


void loop() {
  //read switches
  int switch1 = digitalRead(sw1);
  int switch2 = digitalRead(sw2);
  int switch3 = digitalRead(sw3);
  int switch4 = digitalRead(sw4);

  //BUTTON CODE
  if (switch1 == 1) {
    readnum(number);
  }
  if (switch2 == 1) {
    readnum((int)air);
  }
  if (switch3 == 1) {
    notStarted = 1;
    //previousSecondMillis = 0UL;
  }
  

  //TEMP PROBE CODE
  //read and update current temp
  readSensors();
  mat1.print((int)air);

  //KEYPAD CODE
  // Wait for a key press
  char key = keypad.getKey();
  char printednum = '0';
  // If a key was pressed, display and read the corresponding number
  if (key != NO_KEY) {
    int newnumber = getNumberForKey(key);
    saynum(newnumber);
    if (newnumber == 14) {
      number = 0;
      mat.clear();
    }
    //update the 7 seg number
    else if (number < 1000 && newnumber != 14 && newnumber != 15) {
      number = number * 10 + newnumber;
    }
    mat.print(number);
    //Serial.println(number);
  }
  if (switch4==1){
    mat.clear();
    mat1.clear();
    
  }
  mat.writeDisplay();
  mat1.writeDisplay();

  //TIMER CODE
  //Setting timer
  if (notStarted == 0) {
    int potVal_H = analogRead(POT_PIN_H);
    int potVal_M = analogRead(POT_PIN_M);

    digit_H = map(potVal_H, 1023, 0, 0, 30);
    digit_M = map(potVal_M, 1023, 0, 0, 59);

    startMinute = digit_H;  
    startSecond = digit_M;  
    minutes = startMinute;
    seconds = startSecond;
    matrix.writeDigitNum(0, (minutes / 10));
    matrix.writeDigitNum(1, (minutes % 10));
    matrix.writeDigitNum(3, (seconds / 10));
    matrix.writeDigitNum(4, (seconds % 10));
    if (switch4==1){
    matrix.clear();

  }
    matrix.writeDisplay();
  }

  else {
    // Countdown timer
    //adapted from https://forum.arduino.cc/t/countdown-timer-and-adafruit-4-digit-7-segment-display-w-i2c-backpack/364882/2
    // --------- Run this every second ------------------
    if (millis() - previousSecondMillis >= oneSecond) {
      //update 7 seg per second
      matrix.writeDigitNum(0, (minutes / 10));
      matrix.writeDigitNum(1, (minutes % 10));
      matrix.writeDigitNum(3, (seconds / 10));
      matrix.writeDigitNum(4, (seconds % 10));
      if (switch4==1){
    matrix.clear(); 
  }
      matrix.writeDisplay();
      if (seconds-- == 0) {
        if (minutes == 0) {
          //If timer reaches 0
          notStarted = 0;
          voice.say(sp2_COMPLETE);
          //minutes = startMinute;
          //seconds = startSecond;
          delay(1000);
        } else {
          // rollever from n:00 to (n-1):59
          minutes -= 1;
          seconds = 59;
        }
      }

      previousSecondMillis += oneSecond;
    }
  }
}

//reads a 4 dig number on speaker
void readnum(int number) {
  if (number > 999) {
    saynum((number / 1000) % 10);
  }
  if (number > 99) {
    saynum((number / 100) % 10);
  }
  if (number > 9) {
    saynum((number / 10) % 10);
  }
  if (number > -1) {
    saynum(number % 10);
  }
}

//reads a single number
void saynum(int n) {
  switch (n) {
    case 1:
      voice.say(sp2_ONE);
      return;
    case 2:
      voice.say(sp2_TWO);
      return;
    case 3:
      voice.say(sp2_THREE);
      return;
    case 4:
      voice.say(sp2_FOUR);
      return;
    case 5:
      voice.say(sp2_FIVE);
      return;
    case 6:
      voice.say(sp2_SIX);
      return;
    case 7:
      voice.say(sp2_SEVEN);
      return;
    case 8:
      voice.say(sp2_EIGHT);
      return;
    case 9:
      voice.say(sp2_NINE);
      return;
    case 0:
      voice.say(sp2_ZERO);
      return;
    case 14:
      voice.say(sp3_CLEAR);
      return;
    case 15:
      //readnum();
      return;

    default: return;
  }
}


// Convert the key to a number (casting caused problems)
int getNumberForKey(char key) {
  switch (key) {
    case '0': return 0;
    case '1': return 1;
    case '2': return 2;
    case '3': return 3;
    case '4': return 4;
    case '5': return 5;
    case '6': return 6;
    case '7': return 7;
    case '8': return 8;
    case '9': return 9;
    case 'A': return 10;
    case 'B': return 11;
    case 'C': return 12;
    case 'D': return 13;
    case '*': return 14;
    case '#': return 15;
    default: return 0;
  }
}


//Depreciated Function
void displayNumber(int number) {
  int digits[4];
  digits[0] = number % 10;
  digits[1] = (number / 10) % 10;
  digits[2] = (number / 100) % 10;
  digits[3] = (number / 1000) % 10;
}

// Reads and updates current temp
// adapted from https://forum.arduino.cc/t/another-bbq-thermometer-wifi-enabled-with-push-notifications/225467
void readSensors() {
  float a0 = analogRead(A10);                            // This reads the "voltage" value on A0. Value is actually divided into 1024 steps from 0-1023.
  float v0 = a0 * .0049;                                 // Converts A0 value to an actual voltage (5.0V / 1024 steps = .0049V/step.
  float r0 = (((r2 * vin) / v0) - r2);                   // Calculates resistance value of thermistor based on fixed resistor value and measured voltage
  float logr0 = log(r0);                                 // Natural log of thermistor resistance used in Steinhart-Hart Equation
  float logcubed0 = logr0 * logr0 * logr0;               // The cube of the above value
  float k0 = 1.0 / (A + (B * logr0) + (C * logcubed0));  // Steinhart-Hart Equation to calculate temperature in Kelvin
  float f0 = (1.8 * (k0 - 273)) + 32;                    // Convert temperature to Fahrenheit

  if (isnan(f0))  // If value is not a number, assign an arbitrary value
  {
    air = int(1);
  } else {
    air = f0;  // Otherwise use the calculated value
  }
}