This blog post summarizes our entire journey, where we landed in the end, and what we learned along the way. You can read our other blog posts for a detailed account of our first meeting with Mark, and further explanation of our prototyping process.

The Team: Mark, Katie McTigue, Stefan Orton-Urbina, Nina Yoo

Mark in his home office

What We Built

The Problem

Mark has difficulty taking things out of his oven and moving them over to his countertop. Some of the factors contributing to the problem are the weight of the baking tray, the distance across the kitchen, and the oven door blocking Mark from getting his wheel chair closer to the oven rack.

Mark demonstrating how the oven door blocks him from getting close to the rack

The Solution

We built a rolling cart with a simple machine that moves a baking tray (or anything else that is placed on it) up and down. The cart is sized to fit over the oven rack with the oven door open so that Mark can slide the tray onto the cart without much effort.

The Oven Elevator at a slightly raised position.

The device after we added the final touches of an acrylic cover and handle.

The tapletop, attached to the linear actuators with custom 3D printed braces.

The joystick control and on/off switch.

What Does This Look Like?

The following is a scenario of use based on Mark’s daily life.

Mark is baking asparagus for dinner. He cuts and seasons the dish on his counter, and moves it onto the baking tray. He pulls his oven elevator cart out from under the counter where he stores it, and pushes the joystick up to raise it to the height of the counter. He pulls the tray onto the cart, and pulls it behind his wheelchair over to the oven. He opens to oven door, pulls out the oven rack, and pulls the cart over it. Then he pushes the joystick down to lower to the cart top to the height of the oven tray, and pulls the baking tray onto the rack. He rolls the cart away and closes the oven door.

When the dish is done cooking, he reverses the process to get the tray back to the countertop. Dinner is served without Mark ever needing help from his wife Renée!

Mark pulling the device onto the oven.

Mark moving the device over to his countertop.

How We Got Here

This process was a humbling experience for all of us. The overall trend was that we started with a very ambitious design and it got simpler and simpler as the weeks went on. In the end the big success was that we were able to get our scope down to something that we could execute in the allotted timeframe, and build something that (we believe) will actually solve Mark’s problem.

Scoping

We had a lot of ideas for features that we thought could help Mark. In our early discussions we considered:

• In-and-out motion for the tabletop in addition to the up-and-down motion, so that Mark wouldn’t have to work to pull the tray onto the top.
• Making the whole cart a remote-controlled device that could wheel around as directed, so that Mark would not have to pull/push it across the kitchen.
• Making the control a handheld remote that is separate from the cart.
• A rechargeable battery that can be plugged in.

These are all features that were sacrificed with Mark’s blessing in order to really perfect the core up-and-down functionality, user experience, and durability of the device.

Our First Design

Because the device needed to fit over the oven door and reach the baking tray, our natural inclination was to make a tabletop surface that was connected to the base on one side and open on the other. Our goal was to maximize the ease of rolling the device up to the oven rack.

An early sketch of the one-sided model, with the oven door sandwiched between the top and bottom levels.

We had several ideas for engineering the mechanism that would move the tabletop up and down, including a belt with two small motors. In the end we decided the simplest approach would be to use a linear actuator— a machine with a motor that crates motion in a straight line. The linear actuator would be supported by two rails on either side, hopefully creating enough stability for the platform to stay in place and support the weight of food.

An early attempt at scaling out the device and planning materials.

Prototyping and Problem-Solving

We built two prototypes. The first was a “works-like” prototype: a simple demo of the actuator going up or down as two pushbuttons are depressed. The second was a “looks like” prototype, built our of foamcore to one quarter scale.

Stefan wiring the linear actuator.

The “looks like” prototype. A plastic syringe came in handy for quickly mimicking a linear actuator.

The “works-like” prototype.

We sat down with Mark to let him handle the prototypes and see if he thought we were on the right track. The “looks-like” prototype was very helpful to him in understanding what we had in mind, and he gave us positive feedback.

A Little Crisis and a Big Redesign

After the relative success of our low-fidelity prototypes, we were eager to plan out the details of materials and start building something more high fidelity. At this point we quickly realized that we might be a little over our heads when it came to structural engineering. Getting the top platform stable with support from only one side was a very real challenge with very real risks if we got it wrong; a 400 degree baking tray falling on Mark was a scary thought.

Nina came up with an idea to simplify the design significantly. We would attach two linear actuators directly to the tabletop, one on each side. We would space them far enough apart for the oven door to fit between them when open. The device would be symmetrical, and could be rolled onto the oven from either side.

Nina sketching the oven elevator 2.0.

Mounting the two linear actuators.

Choosing the Controls

Now that we had a game plan for the mechanical components, we could invest some time in choosing the controls for turning the device on and off, and moving the platform up and down. We had a pretty complete picture of Mark’s motor skills with his hands and fingers, because in our first meeting with him we had extensively discussed both how he controlled his wheelchair, and how he used a keyboard and mouse.

Notes on how Mark controls his wheelchair.

In our first meeting with Mark, he explained the symptoms of his acioscapulohumeral muscular dystrophy to us in the context of what he could and couldn’t do comfortably while playing a videogame:

To control the platform, we chose a joystick that closely resembled the one Mark used to control his chair. We mounted it on the side of the cart at a height that Mark could reach comfortably. Because it is attached to the vertical pole of the linear actuator, it is simple to use as “up for up” and “down for down.”

Programming the joystick to control the linear actuators.

Fabrication and Final Touches

Essential components of our final product:

• Custom-cut tabletop made of 1/4 inch plywood
• Wooden base made from 2x4s and 1/4 inch plywood
• 2 braces to attach the tabletop to the actuators (3D printed)
• 1 acrylic cover to protect the electrical components (laser cut)
• 1 joystick and 1 standard on/off switch
• 8 double A batteries and a plastic battery holder (8 batteries will power the device for ~60 minutes of use!)

Stefan soldering.

Stefan testing the actuators post-soldering.

The lasercutting file for the acrylic cover which fits over the base protecting the electrical components.

Conclusions and Lessons Learned

While making the Oven Elevator, we learned a few things in the process. One of the biggest setbacks that we went through, as mentioned before, was the reality of our design and how we were going to build it safely for Mark to use. Instead of taking the time to create a prototype on SolidWorks, or some sort of 3D emulator, earlier, we were more intent on starting to build. This caused us to go back on our designs multiple times due to the severity of budget, safety, and time remaining. We could have possibly saved us time if we spent more time working on fabricating a prototype to get an idea of how it works rather than just aiming to build in one go/ hoping it will work. Another lesson that was learned during the process, is to constantly be able to see from Marks point of view. During the process, we had a lot of design iterations, in drawings, where we struggled to decide on how we should implement a handle for Mark to use. This was more difficult to tackle than we assumed because we could not see Mark often so we needed to remind ourselves to think about how Mark may use it/difficulties for Mark that may occur. However, despite not having to meet with Mark often, we closely noted down what Mark wanted and made sure to fit his needs into our design. This caused us throw away previous ideas that we had, but having Mark give us his opinion gave us a firm direction to follow.

Mark testing out the oven elevator for the first time

Technical Details

Code

/*
   Oven Elevator
   Katie McTigue, Stefan Orton-Urbina, Nina Yoo

   Summary:
   We read the y-axis of a joystick, and move two linear actuators
   up and down in synchrony accordingly. The actuators move at a 
   steady pace for as long as the joystick is pointed in one direction.
   The actuators stop on their own mechanically if they reach the 
   minimum/maximum extension.
   
   Inputs:
   1 Joystick

   Outputs: 
   2 Linear Actuators
*/

//Linear actuator vars
#define EN 9
#define MOTOR1 8
#define MOTOR2 10

#define BUT1 2 
#define BUT2 3

//Joystick vars
const int SPEAKPIN = 2;
const int YPIN = A1;
const int MIDDLEPOINT = 507;
const int MARGINOFERROR = 5;

//Note: X axis pin is not needed, and doesn't even need to be wired

void setup() {
  Serial.begin(9600);
  pinMode(EN,OUTPUT);
  pinMode(MOTOR1,OUTPUT);
  pinMode(MOTOR2,OUTPUT);
}

void loop() {
  //Read Y pin of joystick
  int yVal = analogRead(YPIN);

  //Move up for as long as the joystick is pointed up
  if (yVal < (MIDDLEPOINT - MARGINOFERROR)) {
    Serial.println("Go Up");
    digitalWrite(MOTOR1,LOW);
    digitalWrite(MOTOR2,HIGH);
    digitalWrite(EN,HIGH);
  } 

  //Move down for as long as the joystick is pointed down
  else if (yVal >= (MIDDLEPOINT + MARGINOFERROR)) {
    Serial.println("Go Down");
    digitalWrite(MOTOR1,HIGH);
    digitalWrite(MOTOR2,LOW);
    digitalWrite(EN,HIGH);
  } 

  //OFF
  else {
    digitalWrite(EN,LOW);
  }

}

Schematic and Design Files

Download lasercutting file for acrylic cover.

What We Built:

We built an interactive iPad stand for Richard that allows him to position his iPad in different directions and angles while keeping a log of his different commissions and tracking how long he’s been working on drawings for each one. On the mechanical spectrum, the stand is comprised of a baseboard made of wood, on which a joint that allows the top plate to move up and down, into which a ball and socket joint is screwed, which connects to the top board via a plate screwed into the plastic of the top material. The top board is made of a plastic composite that was milled on both sides, and has slots for Richard’s iPad and Apple Pencil. The electronics consist of a screen, buttons to navigate the menu on the screen, and a rotary encoder to input names for each commission.

Final Images

Photo showing scale and proportion of device from the front.

Overall image showing scale and proportion, and the back of the device.

Close up showing the different pockets and finger slots milled into the front of the board

Up close shot showing the two joints, which connected the base to the top board, in action.

Close up of added acrylic semi circles that could be rotated to keep the iPad from falling out during use. You can also see the paper backing that was glued to the back of where the iPad rests so that the t screws attaching the joint to the board were covered.

Close up of corners on base and top board hand rounded so as to be comfortable to use for a long period of time.

The board can be positioned and different heights and angles, and is adjusted manually, as shown in this photo.

This video shows some of the menu’s functionality, including entering a new project, saving then resuming said project, and then archiving it once it’s finished.

The use for the three buttons depends on which menu screen is being shown. The function of each button is shown above it on the screen.

A rotary encoder was used to input names for each project that was being timed. These names were inputted on the “Start New” screen.

The device can be placed on your lap or on a table, and your iPad rests into the board while you draw. The pocket in the board is milled for Richard’s 12″ iPad, the one being used for reference in this photo is a 9″ iPad.

Here’s a hypothetical situation for its use: Richard and his wife have just finished dinner and now are sitting on the couch watching the news, as they do many nights. Richard wants to work on drawings for a new jewelry commission of a necklace charm he’s just received for a client, Anna, so he grabs his iPad stand, puts it on his lap and begins.

He starts by turning on the screen embedded in the top board of the stand, and navigates to input a new project. Using the rotary encoder, he names the project “Anna”, hits the center button to go navigate to and start the timer, and begins to draw on his iPad. When he’s finished drawing for the night, he hits the “Archive” button, then switches the screen off. The next night, he starts up the timer where it left off and continues to draw Anna’s necklace charm.

How We Got Here:

In our chart we’d hoped to have the electronic and design components prototyped much earlier in the process so we could spend time at the end combining the two, doing lots of testing, and showing a functional MDF prototype to Richard before moving to the final material. We ended up deviating significantly from the timeline we set up in our Gantt chart. Our software was much more complicated and time consuming to write than expected, and it took longer than we’d expected for the mills to be completed due to the number of people using the mill, so we ended up working on both the electronic and design portions of the device up until the end of the project.

Initial sketch I used as a guideline to create the 3D model for milling the back of the board for the electronics to fit into.

It was challenging to update the design for the final mill into the new plastic we were using because of time restraints. Because of the line to use the CNC machine, we needed to submit the final mill file the day after we got the prototype fully milled for it to be finished in time. The other was that we weren’t able to get our electronics wired together as quickly as we’d expected due to software issues, so we weren’t able to mount them into the MDF prototype and actually test all of the components together as we’d initially hoped to.

Notes from measuring the electronics against the MDF prototype.

When we milled the back of the MDF prototype, we did our best to gauge what measurements were correct and which should be changed. I realized it would be difficult to mill at a depth where everything would be submerged in the material, and changed my design for the back to include a larger pocket for the electronics to be drilled into. Because of this, I could no longer plan to just screw an acrylic cover over the components and instead began to plan a vacuum-formed plastic piece that would cover the components.

The plastic composite we were using for our final prototype after being milled, before any hand finishing.

Our functional design/mechanical prototype featuring a baseboard made from scrap wood, but that kept the device from tipping, and a joint that interfaced with the top board and allowed for a great degree of movement for positioning.

Our initial attempt at combining the electronic and design components of the device immediately revealed some problems with fitting the pieces together.

A lot of finishing work went into the design, including rounding all the edges and sanding both the base and the top board, and spray painting the top, making a paper backing to cover the t-screws holding the top board in place, and paper, acrylic, and vacuum formed styrene covers for the electronics. If I’d done this a second time I would have made the top board grey so that it wouldn’t get dirty so quickly.

During crit we had Richard hold the device on his lap as he might at home. From his feedback, we decided to make the baseboard a bit wider, and to round the edges of the wood more for his comfort.

Due to wrong initial dimensions and changing components, some of what was milled needed to be altered. I did this by dremmeling the pockets/holes until they were roughly the desired size/shape. The screen pocket was just an issue of depth for the component, not for its acrylic cover, so I dremmeled a deeper pocket for the component, then used a glue specifically for acrylic to bond the component to it’s cover, and screwed the cover into the plastic.

The screen pocket and hole for the power switch after being dremmeled, before their new components were added back in.

We encountered a lot of issues with the electronics, particularly after we’d solder them all together. We’d initially soldered all of our wires into the Arduino, but after we continued to have problems, we ended up cutting the wires and moving back to sticking the wires into the header pins mounted on the arduino.

When working on developing the display software, we encountered difficulties with the internal arduino timing and the rotary input

After many hours analyzing the code to determine what was causing issues with our rotary input, we determined that it was due to a library conflict, and rewrote the code to utilize a different library- resulting in an encoder that “worjs”.

Developing the software was a lot more difficult than anticipated. Previous to receiving the screen, we focused on creating separate files of code for the logic of each individual function of the device- rotary input, timing, and data storage. Unfortunately, combining perfectly functional separate programs into one doesn’t always work out perfectly; we experienced library and compilation conflicts that caused us to have to re-write large sections of code for it to be compatible with other functions of the device.

Conclusions

While a lot of the feedback we got from the crit revolved around things we already knew were issues – in particular, that “some components weren’t secured/working right”, and that because our ball socket wore out during the crit the top board was “floppy”, some of the feedback brought up things we hadn’t even considered. Someone brought up that we should “consider whether the client was right or left handed” because our current positioning of the electronics was on the left side of the board.

Two people brought up the issue that maybe using an app or timer already on the iPad would be more effective, with someone saying that “the UI for tracking projects was clunky and would be better solved by an app on the iPad”. While this is a good point, and there may be more sophisticated timer options in the app store, this was something we’d talked about with Richard when we’d first met with each other, and while he acknowledged that it was something he could do on his iPad, he liked the idea of it being somewhere he wouldn’t have to minimize his drawing app to access.

It was interesting to work with Richard and to create something that tailored to his needs and interests specifically. Making something for the purpose of a crit is very different from the process of making something to last someone for a while. I think if we were able to get a stronger ball and socket joint, as the one we used wore out incredibly quickly, the mechanical and design aspects of this project would last a good amount of time and be reliable for him to draw on.
We went into this process thinking both the design and the electronic aspects were a lot less complicated than they actually ended up being, and so we didn’t front-load the work as much as we should have. Our design elements ended up needing a lot more finishing work than was initially expected, and because we were handing it off to a client as opposed to it just being for crit, I felt more compelled to do the work to make it look nice and functional. If we were to do this project again knowing what we know now, I think we would make a big effort to frontload as much as possible, and get electronics and a basic design prototype in place as quickly as we could so that we could focus more on testing and making this stable enough to last Richard a long time.

Technical Details

Code Submission

/***************************************************
  Adrien Krupenkin, 2019
  Operates a menu-based system that allows the user to 
  input and track projects they are working on, to be displayed
  on a TFT screen.
 ****************************************************/


#include "SPI.h"
//#include "Adafruit_GFX.h"
#include "Adafruit_ILI9341.h"
#include <Encoder.h>
#include <Fonts/FreeSerif24pt7b.h>
#include <Fonts/FreeSans18pt7b.h>
#include <Fonts/FreeSans12pt7b.h>
const int button1Pin = 5;
const int button2Pin = 6;
const int button3Pin = 7;
const int offPin = 3;
int menuPage = 0;
int currentprojectNumber = 0;
int pastprojectNumber = 0;
int scrollVal = 0;
int positionVal;
bool button1Val = 1;
bool button2Val = 1;
bool button3Val = 1;
bool onoff = 1;
char letter;
const int encoderA = 2; // CLK pin of Rotary Enocoder
const int encoderB = 3; // DT pin of Rotary Enocoder
const int BUT_ENTER = 4; // button pin of Rotary Enocoder
String projects[15];
String pastprojects[32];
String projectTimes[15];
String pastprojectTimes[32];
// variables for rotary encoder
int aLastState;
// didn't implement debouncing
int encoder_pos = 1;
int n = 0;
int seconds[15]={0,0,0,0,0,0,0,0,0,0,0,0,0,0,0};
int minutes[15]={0,0,0,0,0,0,0,0,0,0,0,0,0,0,0};
int hours[15]={0,0,0,0,0,0,0,0,0,0,0,0,0,0,0};
int lim = 0;
unsigned long count;
unsigned long offcount = 0;
bool timer[15];

// letters, numbers, space, characters
const int str_len = 44;
char alphabet[44] = "abcdefghijklmnopqrstuvwxyz0123456789 !?,.:-";
String message = "";

// Declare encoder (from library)
Encoder encoder(encoderA, encoderB);



// For the Adafruit shield, these are the default.
#define TFT_DC 9
#define TFT_CS 10

// Use hardware SPI (on Uno, #13, #12, #11) and the above for CS/DC
Adafruit_ILI9341 tft = Adafruit_ILI9341(TFT_CS, TFT_DC);


void setup() {
  pinMode(offPin, INPUT_PULLUP);
  pinMode(BUT_ENTER, INPUT_PULLUP);
  Serial.begin(9600);
  aLastState = digitalRead(encoderA); //initialize variable for rotary encoder
  Serial.println("ILI9341 Test!");
  pinMode(button1Pin, INPUT_PULLUP);
  pinMode(button2Pin, INPUT_PULLUP);
  pinMode(button3Pin, INPUT_PULLUP);
  tft.begin();
  int encoder_pos;
  tft.fillScreen(ILI9341_BLACK);
  tft.setFont(&FreeSans18pt7b);

}


void loop(void) {

  //Home Screen
  if (menuPage == 0) {
    tft.setRotation(1);
    tft.setCursor(20, 60);
    tft.setTextColor(ILI9341_GREEN);  tft.setTextSize(1);
    tft.setFont(&FreeSerif24pt7b);
    tft.println("Hello Richard!");
    tft.setFont(&FreeSans18pt7b);
    tft.setCursor(35, 115);
    tft.setTextColor(ILI9341_WHITE); tft.setTextSize(1);
    tft.println("What would you");
    tft.setCursor(35, 145);
    tft.println("like to do today?");
    tft.setFont(&FreeSans12pt7b);
    tft.setTextColor(ILI9341_YELLOW);    tft.setTextSize(1);
    tft.setCursor(15, 215);
    tft.println("Resume");
    tft.setCursor(135, 215);
    tft.println("New");
    tft.setCursor(215, 215);
    tft.println("History");
    button1Val = digitalRead(button1Pin);
    button2Val = digitalRead(button2Pin);
    button3Val = digitalRead(button3Pin);
    if (button1Val == 0) {
      menuPage = 1;
      tft.fillScreen(ILI9341_BLACK);
    }
    if (button2Val == 0) {
      menuPage = 3;
      tft.fillScreen(ILI9341_BLACK);
    }
    if (button3Val == 0) {
      menuPage = 4;
      tft.fillScreen(ILI9341_BLACK);
    }
  }

  //Resume Menu
  if (menuPage == 1) {
    tft.setRotation(1);
    tft.setCursor(20, 60);
    tft.setTextColor(ILI9341_GREEN);  tft.setTextSize(1);
    tft.setFont(&FreeSerif24pt7b);
    tft.println("Project Menu");
    if (0 <= n && n <= currentprojectNumber)
    {
      tft.setFont(&FreeSans12pt7b);
      tft.setTextColor(ILI9341_WHITE);    tft.setTextSize(1);
      positionVal = positionVal + 20 - scrollVal;
      tft.setCursor(15, 115);
      tft.print(projects[n]);
      //will later edit to list name and time
    }
    tft.setCursor(15, 215);
    tft.setFont(&FreeSans12pt7b);
    tft.setTextColor(ILI9341_YELLOW);    tft.setTextSize(1);
    tft.println("Previous");
    tft.setCursor(115, 215);
    tft.println("Select");
    tft.setCursor(215, 215);
    tft.println("Next");
    button1Val = digitalRead(button1Pin);
    button2Val = digitalRead(button2Pin);
    button3Val = digitalRead(button3Pin);
    if (button1Val == 0) {
      n = n - 1;
      tft.fillScreen(ILI9341_BLACK);
    }
    if (button2Val == 0) {
      menuPage = 2;
      count = 0;
      tft.fillScreen(ILI9341_BLACK);

    }
    if (button3Val == 0) {
      scrollVal = scrollVal - 25;
      tft.fillScreen(ILI9341_BLACK);
      n = n + 1;
    }
  }

  //Resume Project
  if (menuPage == 2) {
    tft.setRotation(1);
    tft.setCursor(20, 60);
    tft.setTextColor(ILI9341_GREEN);  tft.setTextSize(1);
    tft.setFont(&FreeSerif24pt7b);
    tft.println(projects[n]);
    tft.setFont(&FreeSans18pt7b);
    tft.setCursor(35, 115);
    tft.setTextColor(ILI9341_WHITE); tft.setTextSize(1);

    if (timer[n] == 1) {
      if (count + 1000 < millis()) {
      //timer operation
    
        //delay (1000);
        seconds[n] = seconds[n] + 1;
        if (seconds[n] == 60)
        {
          seconds[n] = 0;
          minutes[n] = minutes[n] + 1;
        }
        if (minutes[n] == 60)
        {
          minutes[n] = 0;
          hours[n] = hours[n] + 1;
        }
        String throwawaystring = " ";
        String colon = ":";
        String timercount = throwawaystring + hours[n] + colon + minutes[n] + colon + seconds[n];
        projectTimes[n] = timercount;
        tft.fillRect(35, 80, 160, 50, ILI9341_BLACK);
        tft.setCursor(35, 115);
        count = millis();
        tft.setTextColor(ILI9341_WHITE); tft.setTextSize(1);
        tft.println(projectTimes[n]);
      }
    }
    tft.setFont(&FreeSans12pt7b);
    tft.setTextColor(ILI9341_YELLOW);    tft.setTextSize(1);
    tft.setCursor(15, 215);
    tft.println("Home");
    tft.setCursor(115, 215);
    tft.println("Timer");
    tft.setCursor(215, 215);
    tft.println("Archive");
    button1Val = digitalRead(button1Pin);
    button2Val = digitalRead(button2Pin);
    button3Val = digitalRead(button3Pin);
    if (button1Val == 0) {
      menuPage = 0;
      tft.fillScreen(ILI9341_BLACK);
    }
    if (button2Val == 0) {
      timer[n] = !timer[n];
      //timer operation
    }
    if (button3Val == 0) {
      menuPage = 4;
      tft.fillScreen(ILI9341_BLACK);
      pastprojectNumber = pastprojectNumber + 1;
      pastprojects[pastprojectNumber] = projects[n];
      pastprojectTimes[pastprojectNumber] = projectTimes[n];
      for (int j = n; j <= currentprojectNumber; j++) {
        projects[j] = projects[j + 1];
        projectTimes[j] = projectTimes[j + 1];
        seconds[j]=seconds[j + 1];
        minutes[j]=minutes[j + 1];
        hours[j]=hours[j + 1];
      }
      currentprojectNumber = currentprojectNumber - 1;
      //save project time and add to list
    }
  }

  //New Project
  if (menuPage == 3) {
    tft.setRotation(1);
    tft.setCursor(20, 60);
    tft.setTextColor(ILI9341_GREEN);  
    tft.setTextSize(1);
    tft.setFont(&FreeSerif24pt7b);
    tft.println("Start New");
    tft.setFont(&FreeSans18pt7b);
    tft.setTextColor(ILI9341_YELLOW);  tft.setTextSize(1);
    //Rotary code here

    int newPos = encoder.read();
    Serial.println(newPos);
    lim = newPos/4 % str_len;
    // lim is the index of the characters in the variable "alphabet"
    if (lim < 0) {
      lim += str_len; // loop around to the first index of the characters, 
      // so you don't have to move all the way back from z to a
    }
    if (encoder_pos != newPos) { // if encoder is being rotated
      //lim = newPos;
      // lim is the index of the characters in the variable "alphabet"
      encoder_pos = newPos;
      char letter = alphabet[lim];
      tft.setFont(&FreeSans18pt7b);
      tft.setTextColor(ILI9341_RED);  tft.setTextSize(1);
      tft.setCursor(100, 100);
      tft.fillRect(100, 75, 30, 40, ILI9341_BLACK);
      tft.print(letter);
      Serial.println("here");
    }
    if (digitalRead(BUT_ENTER) == 0) { // if encoder button is pressed, i.e. to select a char
      delay(500); // so you don't accidentally select the letter many times
      message = message + alphabet[lim]; // add the selected letter to the message
      tft.setCursor(30, 135);
      tft.setTextColor(ILI9341_WHITE);  tft.setTextSize(1);
      tft.println(message);
      Serial.print(message);
    }

    tft.setFont(&FreeSans12pt7b);
    tft.setTextColor(ILI9341_YELLOW);    tft.setTextSize(1);
    tft.setCursor(115, 215);
    tft.println("Enter");
    tft.setCursor(215, 215);
    tft.println("Cancel");
    button1Val = digitalRead(button1Pin);
    button2Val = digitalRead(button2Pin);
    button3Val = digitalRead(button3Pin);
    if (button1Val == 0) {
      //backspace operation
    }
    if (button2Val == 0) {
      menuPage = 1;
      projects[currentprojectNumber] = message;
      message = "";
      currentprojectNumber = currentprojectNumber + 1;
      tft.fillScreen(ILI9341_BLACK);
    }
    if (button3Val == 0) {
      menuPage = 0;
      tft.fillScreen(ILI9341_BLACK);
    }
  }

  //History Menu
  if (menuPage == 4) {
    positionVal = 100 - scrollVal;
    tft.setRotation(1);
    tft.setCursor(20, 60);
    tft.setTextColor(ILI9341_GREEN);  tft.setTextSize(1);
    tft.setFont(&FreeSerif24pt7b);
    tft.println("History");
    for (int y = 0; y <= pastprojectNumber; y++)
    {
      tft.setFont(&FreeSans12pt7b);
      tft.setTextColor(ILI9341_WHITE);    tft.setTextSize(1);
      if (60 < positionVal && positionVal < 215) {
        tft.setCursor(15, positionVal);
        tft.print(pastprojects[y]);
        tft.print(pastprojectTimes[y]);
        //will later edit to list name and time
      }
      positionVal = positionVal + 25;
    }

    tft.setFont(&FreeSans12pt7b);
    tft.setTextColor(ILI9341_YELLOW);    tft.setTextSize(1);
    tft.setCursor(15, 215);
    tft.println("Up");
    tft.setCursor(115, 215);
    tft.println("Home");
    tft.setCursor(215, 215);
    tft.println("Down");
    button1Val = digitalRead(button1Pin);
    button2Val = digitalRead(button2Pin);
    button3Val = digitalRead(button3Pin);
    if (button1Val == 0) {
      scrollVal = scrollVal + 25;
      tft.fillScreen(ILI9341_BLACK);
    }
    if (button2Val == 0) {
      menuPage = 0;
      scrollVal = 0;
      tft.fillScreen(ILI9341_BLACK);
    }
    if (button3Val == 0) {
      scrollVal = scrollVal - 25;
      tft.fillScreen(ILI9341_BLACK);
    }
  }

}

Schematic

Design Files

Linked is the .stl file for milling the top board, as well as .dxf files for lasercutting the screen cover and the semi-circles that hold the iPad in place.

Design Files

Read more about earlier stages in our project:

First meeting/ideation: https://courses.ideate.cmu.edu/60-223/s2019/work/interview-with-elinor/

First prototype: https://courses.ideate.cmu.edu/60-223/s2019/work/team-elinor-prototype-documentation/

What We Built

The “puck” is compact enough to be carried wherever Elinor goes and was designed with her hand size in mind.

A sequence of blue pulses guides the rate of deep breathing.

The ring of lights turns green to signal the end of a minute of deep breathing.

Inspired by smooth river stones, the puck was designed to be smooth and rounded for great ergonomics.

As the portable puck charges in the cable, a display guides Elinor through more mindfulness exercises.

The puck and dock were designed together to be approachable and small enough to be placed on a nightstand or desk.

Fitting all the necessary components inside both enclosures proved a challenge, but was possible through careful modeling and prototyping.

The puck measures about an inch and a half in height, meaning that the many parts inside had to be stacked and arranged carefully.

A pair of copper contacts on the dock provide omnidirectional charging to the battery inside the puck.

Two concentric rings surround the capacitive touch sensor. When Elinor returns home, she can simply drop the puck on the dock to start charging – no need to align contacts or plug in a cable.

The puck charging on the dock.

Here’s how we imagine Elinor using our device:

Elinor wakes up in the morning and opens her calendar app on her phone to familiarise herself with the agenda for the day. The first thing she needs to do is drop her daughter off at school. Before leaving, she takes her mindfulness device with her and drops it in her bag.

After dropping her daughter off, she goes straight to a meeting and then to the grocery store. By the time she finishes grocery shopping, Elinor is feeling exhausted and has 5 minutes to spare before she needs to go back home. She takes out her mindfulness device and completes some deep breathing for 2 minutes while playing calming music on her meditative playlist, and feels so much more relaxed than she did before. She’s ready to go back home and tackle the rest of her day!

When she gets home, she decides she wants to do a few more breathing exercises. She sits at her desk where the dock for her mindfulness device is, and reads the instructions on the screen for 5 more minutes. Before going to sleep, she drops her mindfulness device in the dock that stays on her desk and it starts charging so it’s ready for her to use tomorrow.

When Elinor drops the puck onto the dock to charge, the display welcome her home and guides her through an optional meditation exercise.

How We Got Here

On the day of our final presentation and critique, we were really proud of how everything came together into a working device. Our last prototype is miles away from how our first prototype looked, and incorporated feedback from Elinor regarding features she wanted to see in our device as well as how she wanted it to look and feel.

Starting to work on what our device would look like

The first 3D print of a solid puck shape. We tested out the size of this one before moving on

One of our first considerations was the form factor of our portable device. Since Elinor will be using this device to help her through meditative exercises and to relax, it was important that the device itself evoked a sense of calmness. We were inspired by shapes like worry stones and organic forms such as a Google home that Elinor could easily hold in one hand while she uses it. We went through several different iterations of the form, starting with a compact puck looking shape and tested out the shape and size with Elinor before landing on the final form.

After testing with elinor’s hand, we landed on a size and also measured out our components to make sure everything fit inside

Notes from our consultation with Elinor after our first prototype

Elinor also provided a lot of inspiration for what types of features she wanted to see in our device. In the beginning, we intended on having an LED ring to visualise her heart rate through pulsing as well as a vibrating motor to provide haptic feedback. After meetings however, we learned that Elinor was less interested in features such as haptic feedback and was more interested in actually learning breathing/meditation exercises, so we created a playlist with calming sounds and guided breathing exercises as a companion and also added in an LCD screen in the dock that would teach her exercises and worked on building the most functionality in regard to the LED ring– such as changing colors. The pulsing started out with only basic white LEDs that corresponded with her heart rate. Elinor told us that she would love to see a variety of colors and that she would prefer colors other than white, so we used a light purple color for the first 5 seconds of usage while a baseline rate is being set, blue during the actual minute of deep breathing, and green to indicate when a minute has passed and the heart rate sensor turns off. We considered what types of colors would be most calming– we started out having red as the indication that a minute has passed, but decided it was too harsh and changed to red.

The idea for a one-minute timer also came from Elinor, who did not previously meditate or complete any type of breathing exercises during her lifestyle. She also did not want 5 or 10 minute long cycles because those are harder to fit into her schedule. As we hope our device can act as a sort of gateway into bringing more relaxation into her life, each breathing cycle is set at one minute at a time so that it’s something quick that can be done even while she is in her car.

We ran into many challenges along the way in regards to the electronic component of building our device. The heart rate sensor that was available to check out in our lab was a really finicky part and often displayed strange (too high or too low) heart rate readings when we used it (more details about the problems we ran into can be read here). We thought about going a different route and pivoting to use a different part or tackle the issue of mindfulness in a different way but ultimately decided to stick with it and order a new heart rate sensor that would hopefully work better. Fortunately, the new part that we ordered behaved much more normally and although far from medical grade, displayed normal and pretty accurate heart rate readings when compared with an apple watch.

Another issue we ran into was with the wireless data transmission. Initially, one of the features that we wanted to have was wireless data transmission from Elinor’s portable device to the dock that stayed at home to display average BPM and average usage amounts over the course of a day and over the course of a week. We worked really hard on implementing this feature and was able to make it work in the beginning stages, but when we went to put all of the parts together, we discovered that both devices needed to use the same specialised pin (the SPI pin) on the Arduino. We attempted to find workaround solutions and followed tutorials that had the same problem but ultimately was not able to fix the issue and decided to devote the rest of our time to making all of the other components of our final product work properly and look great.

We added in a capacitative sensor that will turn on and off the device so that the heart rate sensor is not constantly measuring

Soldering all of our components together to make them fit into our shell

Making sure everything fits inside!

Working on the contacts to make sure the rechargeable battery works

Working hard to debug and work with the mess of components on the 11th hour!

As most long-term projects go, we were initially able to stick to our schedule and plan as we worked individually on software and hardware. However, as we got closer to final critique day, we realised that fabrication and assembly of our products took much, much longer than we expected. The night before the critique we worked tirelessly to debug, solder, and assemble everything into a functioning beautiful device! The issues that we mentioned were by far our biggest problems but we wanted to make sure to deliver Elinor something that she would really appreciate.

Conclusions and Lessons Learned

At the critique, the positive feedback was largely directed at the form factor of the device and how polished it looked– “The design of the device is lovely. It looks very touchable and smooth.” “Nice design and good calming indicator.” We were really happy that visitors who came by and interacted with our device seemed to feel a sense of calmness from the device itself, which is what we had really hoped and intended for. 

Recurring feedback that we received mostly revolved around if Elinor would actually use our device, and the necessity of it– “It depends on the user to create a habit.” “Doesn’t seem to solve a large problem.” Some future features we would want to add to address this sort of feedback would be to focus on data feedback between the portable device and the dock so that Elinor can take a look back at how many times she has used the device, and if her resting heart beat or average BPM has improved over the course of her usage. This would definitely be implemented in a second iteration of our project. 

 We also considered this “can’t she just use her phone?” factor throughout our design process, and made a conscious decision to keep our device as analog as possible. Elinor uses her digital phone, tablet, and watch very often and we wanted to create something more tactile and interactive that she can integrate into her daily life without feeling a huge sense of foreignness. The functionality exists on other devices, but the way that the features are presented is one of the most important parts.

In a future iteration, we would also definitely add more features for the portable component such as a different time length setting for the device so that Elinor can have 1, 3, 5, 10 minute settings. Some other, more physical features we would focus on if we had more time would be to refabricate the dock using a more sturdy print to make sure that it will stay on her desk without any issues or breakage. 

Working with Elinor was a really unique experience that we had not been able to experience through a class or through any other project. At school we are constantly surrounded by like-minded people who often have similar life experiences. We had a great time visiting Elinor’s house and learning about the different perspectives that she has on life. This project also allowed for us to stretch our minds in terms of designing for her, since the prompt was really open-ended and let us work with an abstract problem like mindfulness.

We did feel a bit of a disadvantage as a two-person team, since one of us almost solely focused on software and the other on hardware. If we had one more person, we feel that we would have been able to add more functionality than we were able to, or spend even more time honing in on the form factor. Despite this, we are really proud of what we were able to create and learned a lot not just about electronics, but about working with people who are different from us! Watching Elinor use our device on final critique day was definitely the highlight of the project.

Schematic:

Code for the portable device:

/* 
 *  Mindfulness Device
 *  
 *  Karen Kim + Ian Shei 
 *  
 *  This code uses a pulse sensor to measure heart rate
 *  and display it through an LED ring. There are 
 *  different thresholds that determine the speed
 *  of the pulsing of the LED ring. 
 *  
 *  Input: heart rate sensor (A0), capacitative sensor (Pin 6) 
 *  Output: LED ring (Pin 5) 
 *  
 *  Example code used from: 
   http://pulsesensor.com/pages/pulse-sensor-amped-arduino-v1dot1
   Version 1.0 by Mike Barela for Adafruit Industries, Fall 2015
*/

#include <Adafruit_NeoPixel.h>    // Library containing
#include <SPI.h>
#include <nRF24L01.h>
#include <RF24.h>

#define CE_PIN   9
#define CSN_PIN 10

// Behavior setting variables
int pulsePin = A0;                 // Pulse Sensor purple wire connected to analog pin 0
int blinkPin = 13;                // Digital pin to blink led at each beat
int ringPin  = 5;                 // pin to do fancy neopixel effects at each beat
int fadeRate = 0;                 // used to fade LED on with PWM on ringPin
int sensorPin = 6;
const int motorPin = 3;

// array for LED brightness pulsing values 
int brightnessVal [201] = {0, 0, 0, 1, 1, 2, 2, 3, 4, 5, 6, 8, 9, 10, 12, 14, 16, 18, 20, 22, 24, 27, 29, 32, 35, 37, 40, 43, 46, 49, 53, 56, 59, 63, 66, 70, 73, 77, 81, 84, 88, 92, 96, 100, 104, 108, 112, 116, 119, 123, 128, 132, 136, 139, 143, 147, 151, 155, 159, 163, 167, 171, 174, 178, 182, 185, 189, 192, 196, 199, 202, 206, 209, 212, 215, 218, 220, 223, 226, 228, 231, 233, 235, 237, 239, 241, 243, 245, 246, 247, 249, 250, 251, 252, 253, 253, 254, 254, 255, 255, 255, 255, 255, 254, 254, 253, 253, 252, 251, 250, 249, 247, 246, 245, 243, 241, 239, 237, 235, 233, 231, 228, 226, 223, 220, 218, 215, 212, 209, 206, 202, 199, 196, 192, 189, 185, 182, 178, 174, 171, 167, 163, 159, 155, 151, 147, 143, 139, 136, 132, 128, 123, 119, 116, 112, 108, 104, 100, 96, 92, 88, 84, 81, 77, 73, 70, 66, 63, 59, 56, 53, 49, 46, 43, 40, 37, 35, 32, 29, 27, 24, 22, 20, 18, 16, 14, 12, 10, 9, 8, 6, 5, 4, 3, 2, 2, 1, 1, 0, 0, 0};

const byte slaveAddress[5] = {'R','x','A','A','A'}; // for radio 
RF24 radio(CE_PIN, CSN_PIN); // Create a Radio

char message[10]; // array length for radio message 
char txNum = '0';

unsigned long currentMillis;
unsigned long prevMillis;
unsigned long txIntervalMillis = 1000; // send once per second


// these variables are volatile because they are used during the interrupt service routine
volatile int BPM;                   // used to hold the pulse rate
volatile int Signal;                // holds the incoming raw data
volatile int IBI = 600;             // holds the time between beats, the Inter-Beat Interval
volatile boolean Pulse = false;     // true when pulse wave is high, false when it's low
volatile boolean QS = false;        // becomes true when Arduoino finds a beat.

// Set up use of NeoPixels
const int NUMPIXELS = 24;           // Put the number of NeoPixels you are using here
const int BRIGHTNESS = 60;          // Set brightness of NeoPixels here
Adafruit_NeoPixel strip = Adafruit_NeoPixel(24, ringPin, NEO_GRBW + NEO_KHZ800);

unsigned long startMillis;
bool inTrial = false;
bool inWait = false;
bool turnOn = false;

int bpmCount = 0;
int bpmAvg = 0;
int totalBPM = 0;
int dailyBPM[10] = {-1,-1,-1,-1,-1,-1,-1,-1,-1,-1};

void setup(){
  pinMode(blinkPin,OUTPUT);         // pin that will blink to your heartbeat!
  pinMode(motorPin, OUTPUT);
  pinMode(sensorPin, INPUT);
  Serial.begin(9600);           // Serial output data for debugging or external use
  strip.begin();
  strip.setBrightness(BRIGHTNESS);
  for (int x=0; x < NUMPIXELS; x++) {  // Initialize all pixels to 'off'
     strip.setPixelColor(x, strip.Color(0, 0, 0));
  }
  strip.show();                     // Ensure the pixels are off
  delay(1000);                      // Wait a second
  interruptSetup();                 // sets up to read Pulse Sensor signal every 2mS

   radio.begin();
   radio.setDataRate( RF24_250KBPS );
   radio.setRetries(3,5); // delay, count
   radio.openWritingPipe(slaveAddress);
}

void loop(){
  int sensorRead = digitalRead(sensorPin);
  // Serial.println(sensorRead);

  if (sensorRead == HIGH){ // use capaacitative sensor to turn on device 
    turnOn = true;
  }

  if (turnOn == true){
  if (inWait == false and inTrial == false and QS == true){  // Quantified Self flag is true when arduino finds a heartbeat
//    analogWrite(motorPin, 100);
     Serial.println("START WAIT"); // setting 5 second baseline for heartrate 
     inWait = true;
//     inTrial = true;
     startMillis = millis();
//     fadeRate = 255;                  // Set 'fadeRate' Variable to 255 to fade LED with pulse
     Serial.println(BPM);
//     sendDataSerial('B',BPM);       // send heart rate with a 'B' prefix
//     sendDataSerial('Q',IBI);       // send time between beats with a 'Q' prefix
     QS = false;                      // reset the Quantified Self flag for next time
  }

  else if (inWait == true and millis() - startMillis >= 5000){
      Serial.println("END WAIT");

      inTrial = true;
      inWait = false;
      startMillis = millis();
    }

  else if (inTrial == true and QS == true){
    bpmAvg = totalBPM/bpmCount;
//    Serial.println(bpmAvg);
    Serial.println("IN TRIAL");
//    Serial.println(millis() - startMillis);

    if (millis() - startMillis >= 5000){
      Serial.println("END OF TRIAL");
      inTrial = false;
      for (int x=0; x < NUMPIXELS; x++) {  // Initialize all pixels to 'off'
        strip.setPixelColor(x, strip.Color(255, 0, 0)); // blue light for during 1 minute cycle
      }
      strip.show();
      send(bpmAvg);
      totalBPM=0;
      bpmCount=0;
      delay(1000);
      turnOn = false;
    }
 //   fadeRate = 400;
    QS = false;
  }

  if (inTrial == true){
    Serial.print("BPM=");
    Serial.println(BPM);
    totalBPM+=BPM;
    bpmCount+=1;
    ledFadeToBeat(BPM);                    // Routine that fades color intensity to the beat
  }

  else if (inWait == true) {
    for (int x=0; x < NUMPIXELS; x++) {  // Initialize all pixels to 'off'
      strip.setPixelColor(x, strip.Color(0, 255, 0)); // turns off after 1 minute
      }
      strip.show();
  }

  else {
    setStrip(255,0,0,BPM);
    delay(3000);
  }
  delay(100);                          //  take a break
}
}

void ledFadeToBeat(int BPM) {
  static int i = 100;
           // Set LED fade value
  if (BPM < 50){
    i+=1;
      if (i >= 201) {
        i = 0;
    }
    fadeRate = brightnessVal[i];
    fadeRate = constrain(fadeRate,0,255);   // Keep LED fade value from going into negative numbers
    setStrip(0,0,fadeRate, BPM);
  }
  else if (BPM >= 50 and BPM < 80){ // setting BPM thresholds to set pulsing speed 
    i+=2;
    if (i >= 201) {
       i = 0;
    }
    fadeRate = brightnessVal[i];
    fadeRate = constrain(fadeRate,0,255);   // Keep LED fade value from going into negative numbers
    setStrip(0,0,fadeRate, BPM);
  }
  else if (BPM >= 80 and BPM < 100){
    i+=3;
    if (i >= 201) {
       i = 0;
    }
    fadeRate = brightnessVal[i];
    fadeRate = constrain(fadeRate,0,255);   // Keep LED fade value from going into negative numbers
    setStrip(0,0,fadeRate, BPM);
  }
  else if (BPM >= 100 and BPM < 120){
    i+=4;
    if (i >= 201) {
       i = 0;
    }
    fadeRate = brightnessVal[i];
    fadeRate = constrain(fadeRate,0,255);   // Keep LED fade value from going into negative numbers
    setStrip(0,0,fadeRate, BPM);
  }
  else if (BPM >= 120 and BPM < 150){
    i+=5;
    if (i >= 201) {
       i = 0;
    }
    fadeRate = brightnessVal[i];
    fadeRate = constrain(fadeRate,0,255);   // Keep LED fade value from going into negative numbers
    setStrip(0,0,fadeRate, BPM);
  }
  else{
    for (int x=0; x < NUMPIXELS; x++) {  // Initialize all pixels to 'off'
       strip.setPixelColor(x, strip.Color(0, 0, 0));
    }
    strip.show();
    i=0;
  }
//  Serial.print("i=");
//  Serial.println(i);
//    sendDataSerial('R',fadeRate);
}

void sendDataSerial(char symbol, int data ) {
//    Serial.print(symbol);                // symbol prefix tells Processing what type of data is coming
//    Serial.println(data);                // the data to send culminating in a carriage return
}

void setStrip(int r, int g, int b, int BPM) {     // Set the strip to one color intensity (blue)
  for (int x=0; x < NUMPIXELS; x++) {
    strip.setPixelColor(x, strip.Color(0, 0, b));
  }
   strip.show();
}

/*
  sending the 10 most recent bpm values to receiver

*/
void send(int bpmAvg) { // function for radio 

  // shift every BPM value to the right for the new bpmAvg
  for (int i=0; i<9; i++){
    dailyBPM[i+1] = dailyBPM[i];
  }
  dailyBPM[0] = bpmAvg; //putting newest bpmAvg value at the front of array

  bool rslt = radio.write( &dailyBPM, sizeof(dailyBPM) );
        // Always use sizeof() as it gives the size as the number of bytes.
        // For example if dataToSend was an int sizeof() would correctly return 2

    Serial.print("Data Sent ");
    for (int x = 0; x < (sizeof(dailyBPM) / sizeof(dailyBPM[0])); x++) {
      Serial.println(dailyBPM[x]);
    }
    if (rslt) {
        Serial.println("  Acknowledge received");
        memset(dailyBPM, -1, sizeof(dailyBPM));

        // for (int x = 0; x < sizeof(dailyBPM) / sizeof(dailyBPM[0]); x++)
        // updateMessage();
    }

    else {
        Serial.println("  Tx failed");
    }
}

Code for the dock:

#include <Adafruit_GFX.h>    // Core graphics library
#include <Adafruit_ST7735.h> // Hardware-specific library for ST7735
#include <Adafruit_ST7789.h> // Hardware-specific library for ST7789
#include <SPI.h>


#ifdef ADAFRUIT_HALLOWING
  #define TFT_CS        39 // Hallowing display control pins: chip select
  #define TFT_RST       37 // Display reset
  #define TFT_DC        38 // Display data/command select
  #define TFT_BACKLIGHT  7 // Display backlight pin
#else
 
  #define TFT_CS        10
  #define TFT_RST        9 // Or set to -1 and connect to Arduino RESET pin
  #define TFT_DC         8
#endif

Adafruit_ST7735 tft = Adafruit_ST7735(TFT_CS, TFT_DC, TFT_RST);

const int CHARGE_PIN1 = A1; // display turns on when contact is made from the device to the dock
const int CHARGE_PIN2 = A2;
const int CHARGE_TOLERANCE = 20;


void setup(void) {
#ifdef ADAFRUIT_HALLOWING
  tft.initR(INITR_HALLOWING);        // Initialize HalloWing-oriented screen
  pinMode(TFT_BACKLIGHT, OUTPUT);
  digitalWrite(TFT_BACKLIGHT, HIGH); // Backlight on
#else
  // Use this initializer if using a 1.8" TFT screen:
  tft.initR(INITR_BLACKTAB);      // Init ST7735S chip, black tab
#endif

  tft.setRotation(1); // rotate the screen to horizontal 
  pinMode(CHARGE_PIN1, INPUT); 
  pinMode(CHARGE_PIN2, INPUT);
  Serial.begin(9600);
  
  testfillcircles(10, ST77XX_BLUE);
  testdrawcircles(10, ST77XX_WHITE);
}

void loop() {
  static bool plugged = false;
  int chargeRead1 = analogRead(CHARGE_PIN1); // reading if contact has been made 
  int chargeRead2 = analogRead(CHARGE_PIN2);
//  Serial.println(chargeRead1);
//  Serial.println(chargeRead2);

  if ((chargeRead1 - CHARGE_TOLERANCE <= chargeRead2) and  // portable device not plugged in 
    (chargeRead2 <= chargeRead1 + CHARGE_TOLERANCE)) {
      plugged = false;  
      tft.fillScreen(ST77XX_BLACK);
     }

  else if (plugged == false) { // this condition fires once we plug in 
    tftPrintTest();
    plugged = true; // portable device marked as "plugged in" now
}
  
  delay(500);
}


void tftPrintTest() { // the content of the screens 
  tft.fillScreen(ST77XX_MAGENTA);
  tft.setTextColor(ST77XX_WHITE);
  tft.setCursor(0, 30);
  tft.setTextSize(3);
  tft.println("WELCOME");
  tft.setCursor(0, 60);
  tft.println("HOME");
  tft.setCursor(0, 90);
  tft.println("ELINOR!");
  delay(3000);
  tft.setTextSize(1);
  tft.setCursor(0, 20);
  tft.fillScreen(ST77XX_BLACK);
  tft.setTextColor(ST77XX_WHITE);
  tft.println("Hello Elinor!");
  tft.setCursor(0, 40);
  tft.println("Hope you are having");
  tft.setCursor(0, 60);
  tft.println("a great day.");
  delay(5000);
  tft.fillScreen(ST77XX_BLUE);
  tft.setTextColor(ST77XX_WHITE);
  tft.setCursor(0, 10);
  tft.setTextSize(2);
  tft.println("Here are some");
  tft.setCursor(0, 30);
  tft.println("breathing");
  tft.setCursor(0, 50);
  tft.println("tips for you!");
  delay(3000);
  tft.fillScreen(ST77XX_BLUE);
  tft.setTextColor(ST77XX_WHITE);
  tft.setCursor(0, 10);
  tft.setTextSize(2);
  tft.println("First, take a");
  tft.setCursor(0, 30);
  tft.println("long, slow");
  tft.setCursor(0, 55);
  tft.println("INHALE.");
  delay(3000);
  tft.fillScreen(ST77XX_BLUE);
  tft.setTextColor(ST77XX_WHITE);
  tft.setCursor(0, 10);
  tft.setTextSize(2);
  tft.println("Next, a quick");
  tft.setCursor(0, 30);
  tft.println("and powerful");
  tft.setCursor(0, 55);
  tft.println("EXHALE.");
  delay(3000);
  tft.fillScreen(ST77XX_BLUE);
  tft.setTextColor(ST77XX_WHITE);
  tft.setCursor(0, 10);
  tft.println("Take a total");
  tft.setCursor(0, 30);
  tft.println("of 10 breaths");
  delay(3000);
  tft.fillScreen(ST77XX_WHITE);
  testfillcircles(10, ST77XX_BLUE);
  testdrawcircles(10, ST77XX_WHITE);
  delay(4000);
  testlines(ST77XX_YELLOW);
  delay(2000);
  testfastlines(ST77XX_RED, ST77XX_BLUE);
  delay(2000);
  tft.fillScreen(ST77XX_GREEN);
  tft.setTextColor(ST77XX_WHITE);
  tft.setCursor(0, 10);
  tft.setTextSize(2);
  tft.println("Here's a");
  tft.setCursor(0, 30);
  tft.println("tip for you!");
  delay(3000);
  tft.fillScreen(ST77XX_GREEN);
  tft.setTextColor(ST77XX_WHITE);
  tft.setCursor(0, 10);
  tft.setTextSize(2);
  tft.println("Deep breaths");
  tft.setCursor(0, 30);
  tft.println("increase your");
  tft.setCursor(0, 50);
  tft.println("oxygen flow.");
  delay(2000);
  tft.fillScreen(ST77XX_GREEN);
  tft.setTextColor(ST77XX_WHITE);
  tft.setCursor(0, 10);
  tft.setTextSize(2);
  tft.println("Also, it");
  tft.setCursor(0, 30);
  tft.println("promotes a");
  tft.setCursor(0, 50);
  tft.println("state of");
  tft.setCursor(0, 70);
  tft.println("CALMNESS.");
  delay(5000);
  tft.fillScreen(ST77XX_WHITE);
  testfillcircles(10, ST77XX_BLUE);
  testdrawcircles(10, ST77XX_WHITE);
}

void testfillcircles(uint8_t radius, uint16_t color) { //graphics for elinor to view while breathing
  for (int16_t x=radius; x < tft.width(); x+=radius*2) {
    for (int16_t y=radius; y < tft.height(); y+=radius*2) {
      tft.fillCircle(x, y, radius, color);
    }
  }
}

void testdrawcircles(uint8_t radius, uint16_t color) {
  for (int16_t x=0; x < tft.width()+radius; x+=radius*2) {
    for (int16_t y=0; y < tft.height()+radius; y+=radius*2) {
      tft.drawCircle(x, y, radius, color);
    }
  }
}

void testlines(uint16_t color) {
  tft.fillScreen(ST77XX_BLACK);
  for (int16_t x=0; x < tft.width(); x+=6) {
    tft.drawLine(0, 0, x, tft.height()-1, color);
    delay(0);
  }
  for (int16_t y=0; y < tft.height(); y+=6) {
    tft.drawLine(0, 0, tft.width()-1, y, color);
    delay(0);
  }

  tft.fillScreen(ST77XX_BLACK);
  for (int16_t x=0; x < tft.width(); x+=6) {
    tft.drawLine(tft.width()-1, 0, x, tft.height()-1, color);
    delay(0);
  }
  for (int16_t y=0; y < tft.height(); y+=6) {
    tft.drawLine(tft.width()-1, 0, 0, y, color);
    delay(0);
  }

  tft.fillScreen(ST77XX_BLACK);
  for (int16_t x=0; x < tft.width(); x+=6) {
    tft.drawLine(0, tft.height()-1, x, 0, color);
    delay(0);
  }
  for (int16_t y=0; y < tft.height(); y+=6) {
    tft.drawLine(0, tft.height()-1, tft.width()-1, y, color);
    delay(0);
  }

  tft.fillScreen(ST77XX_BLACK);
  for (int16_t x=0; x < tft.width(); x+=6) {
    tft.drawLine(tft.width()-1, tft.height()-1, x, 0, color);
    delay(0);
  }
  for (int16_t y=0; y < tft.height(); y+=6) {
    tft.drawLine(tft.width()-1, tft.height()-1, 0, y, color);
    delay(0);
  }
}

void testfastlines(uint16_t color1, uint16_t color2) {
  tft.fillScreen(ST77XX_BLACK);
    for (int16_t y=0; y < tft.height(); y+=5) {
      tft.drawFastHLine(0, y, tft.width(), color1);
      }
    for (int16_t x=0; x < tft.width(); x+=5) {
      tft.drawFastVLine(x, 0, tft.height(), color2);
      }
}

3D fabrication files:

Link to A360

For our final project, our team was paired with an older friend, Irv, to develop a “convenience machine” he could use to improve his life. Over the course of a month, we interviewed him on his daily activities, developed a prototype, and refined a final model that Irv could take home and use. By employing methods of Physical Computing, including programming, circuitry, and physical fabrication, we created an “interactive itinerary” device unique to Irv’s busy lifestyle.

What we built

The Interactive Itinerary is a digital reminder system (think Google Calendar) that feels like an analog device. The user can set alarms for tasks or events without having to type on a small and sensitive touchscreen. Instead, the user can write down the task on a piece of paper and set the alarm using built in knobs.
The interactive itinerary prototype has 3 key components: 1) Two knobs that set the task and alarm 2) the digital display that shows task, time, and set time 3) a vibrating motor that acts as an alarm. To use the interactive itinerary, the user writes down a task on a piece of paper with an empty list from 1 – 10. Using the task knob, the user can turn to the desired task number turn the alarm on. Then, they would set the time using the time knob. When the alarm starts to buzz, the user can turn off the alarm with the task knob.

Outer housing of itinerary is bound with elastic. The knobs and buttons are protected by the sides but still accessible without having to open the device.

Inside of itinerary displays an LCD screen, casing for electronics, and notepad.

“Task” and “time” knobs for programming the device, indicator LED, and sliding mechanism to access electronics.

The top side of the electronics housing reveals the on/off button, “snooze” button for offsetting an alarm, and charging port.

A small manual is mounted under the notepad for troubleshooting and quick reference.

Sliding mechanism allows access to RTC for battery changes, arduino reset button, and any other troubleshooting issues.

How it’s used:

On a normal day, Irv wakes up and unplugs the device from the charging port. On the notepad, he writes down his tasks for the day and the time they occur (i.e., medications, tennis meetings, cultural events). He can then use the knobs on the device to set reminders for the tasks he would like to be reminded of. Irv then closes the journal and takes it with him throughout the day. When a reminder goes off, the journal will buzz and flash, and the task will appear on screen. He can then press a button to turn off the alert, or push “snooze” to delay the reminder for another 15 minutes.

How we got here

Following the formative critique phase of this project, our team created a list of milestones we intended to hit for our final meeting with Irv. After showcasing our first prototype at the formative critique to Irv, we learned that a 15 minute snooze button would be a useful addition to the device. We learned that Irv preferred to set alarms in military time and would like an on/off switch on the device to conserve power.

We created a to do list with Irv to ensure that the final product suits his needs and ranked it based on importance.

The snooze button and on/off button were relatively easy to set up. The most important part about the snooze and on/off button is picking a button that has the right tactile feeling. After the formative critique, we went to the physical computing lab and pulled out many button options for Irv to test. He settled on a red momentary push button for the snooze button and a black latching push button for the on/off switch.

Irv tests different buttons for the new features on his interactive itinerary.

The next thing we wanted to add to the interactive itinerary is displaying the task “home page” when the alarm for that task goes off. This feature, which wasn’t in our first iteration, makes sense because when the alarm goes off, we would want to know which task alarm was going off. So, when the alarm goes off now, the task number that is ringing is also displayed on the LCD. Without this feature, the user would have to look at the written list of tasks and check each alarm time to see which task was going off.

In order to manage all our coding iterations and keep track of our goals, we wrote comments on our new versions to communicate what was working, what wasn’t, and what our next steps should be.

To make the device portable, we intended to make the device battery powered. We ordered a lithium ion battery and a chip so that Irv can charge the device at night. The chip ensures that the battery charges without overcharging the lithium ion battery. After setting up the battery, we soldered the on/off switch to the battery, which acts as an on/off switch for the device. SInce the lithium ion battery is quite large, our goal to make the device smaller was getting harder.

Once all the necessary components were bought and collected, we laid out the device components to determine the best way to stack the device to minimize the space. As shown in the picture below, the device would already be very cramped, even before soldering everything together. Since we were no longer considering bootloading our device onto a smaller device, we had to stick to the arduino uno. So at this point in the process, our goal to make the device “pocket-sized” was becoming less and less feasible.

In an ideal set up, the LCD screen would sit at the top of the device. The charging port is in the top left of the box. The snooze button and the on/off button would rest on the top, closer to the spine of the box.

While we knew what the general layout of our itinerary would be, we explored ways we could configure the electronic and tactile components to save space. We also thought of ways we could construct a sliding mechanism with lasercut material that would allow the electronics to be accessible without force.

While designing the prototype for the device casing, we considered several materials for the electronic housing. Acrylic, while sturdy and aesthetically pleasing, is prone to shattering. So if Irv decided to put this device in his back pocket, or applied too much pressure to it, it might crack under pressure, rendering the casing useless. Styrene is a plastic that is a bit more flexible but sturdy as well. Ultimately, styrene would not have been ideal to use as our electronic casing because we needed to make a case that had precise cuts and we would not be able to laser cut styrene. So, we decided that ⅛” birch plywood was the best material to use for the device. We also decided to include a sliding lid on the electronic casing so that if Irv needed to replace the RTC battery or reset the arduino to troubleshoot.

The sliding mechanism consisted of two pieces of plywood glued together that fit into the top facing of the case. To keep the lid on, a support bar was built under the top facing. That way, the lid can rest on the support bar.

In addition, the laser cutters at TechSpark burn more, cut slower, and are more volatile than the laser cutters in IDeATe. Luckily, the first print was just a prototype; so we planned on printing the final casing in IDeATe. The first prototype was also larger than we needed. There was a lot of room inside the casing. We were able to make the case thinner and narrower. While the notebook would still not be pocket-sized, it was still considerably smaller.

After considering different methods of fabrication for the outer casing, we decided traditional “bookbinding” techniques would best fit Irv’s aesthetic. Inspired by moleskine journals, we used flexible chipboard, faux leather, and elastic to create a housing that would be sturdy, comfortable, and stylish.

Conclusion

After the final crit, we added one last feature to our box because of a comment made to “make the alarm stronger” if it couldn’t be kept in his pocket. Since we were planning to make the box small, we didn’t account for possible issues if it was too large to fit into a pocket. Some of these issues would be would Irv notice the alarm going off if it wasn’t on his person. Our solution was to add a small LED next to the task and time knobs that would blink at the same rate that the buzzer would be going off. This would allow Irv to see the alarm going off as well as feel it. As he’s told us that his hearing isn’t the best, we thought we’d try to stimulate his other senses with the alarm.

Working with Irv was more than just an opportunity to gain insight on the life of an older person, but a beneficial experience in collaborating with a client. Not only did learning about Irv’s lifestyle allow us to empathize with his daily needs, it also taught us how to iterate and design under constraints that can assist virtually anyone. While the itinerary was developed for his particular lifestyle and activities, all of us can benefit from a method of keeping track of lists with time notifications. In principle, this project was a great introduction to universal design. By solving the problems of overlooked populations rather than the “average person,” we can innovate more effectively and inclusively.

Moreover, we believe that having a person to design for other than ourselves pushed us further to create something functional and intuitive. Since we realized Irv wouldn’t have the means to fix the RTC if it reset, for example, we decided late into the project to build in a time-setting function into the program. This mindset also prompted us to add in a sliding compartment to access the electronics, a “sleep” mode to save power, and a reference manual for troubleshooting. If we hadn’t had an actual client to work with, it would have been easier to let these considerations slide when you know you can can quickly reupload, solder or glue something back together. Knowing that Irv will be keeping this in his home and hopefully trying it out, we feel more confident knowing that we built in features to extend the project’s lifespan.

Code and Files

schematic

Code

/*  Reminder - Final Project
    Eliza Pratt
    Helen Yu
    Eric Mendez

***********************************************************
description:
  Use "Task" and "Time" button knobs to set alarms throughout the day. The LCD display shows
  the current task (1 to 9) and the set alarm time. If the task button is pushed, the alarm
  is turned on. There is a snooze button as well, so if you'd like to be snooze the alarm, the
  button adds 15 min to the on-going alarm. The display will reflect that the alarm was snoozed.
  

Pin mapping:
  - The task and time knobs have 3 input pins each and those are pins 2-7
  - The snooze button is 8
  - The buzzer pin is 9
  - The pin for the LED is 11
  - The SCA/SCL pins are used on both the LCD screen and RTC (real time clock)
  - If adding a Lipo battery and power regulating chip (see our schematic)
    - use a switch to interupt the connection from the chip to Vin on the Arduino

example code used:
  - Used examples from TimeLib, LiquidCrystal, DS1307RTC, and Encoder to help set up
  their usage in this project.

***********************************************************
*/
#include <Wire.h>
#include <TimeLib.h>
#include <LiquidCrystal_I2C.h>
#include <DS1307RTC.h>
#include <Encoder.h>

// set up lcd
LiquidCrystal_I2C screen(0x27, 16, 2);

// time elements
tmElements_t tm;

Encoder taskKnob(2, 4);
Encoder timeKnob(3, 6);

// task and time knobs / buttons
const int TASK_BUTTON = 5; // yellow wire
const int TIME_BUTTON = 7; // yellow wire
const int SNOOZE_BUTTON = 8; // white wire
const int MOTORPIN = 9;
const int LEDPIN = 11;

long oldTaskPosition  = -999;
long oldTimePosition  = -999;

unsigned long motorTimer = 0;
unsigned long screenLight = 0;

bool motorState = false;
bool oldTimeState = false; // stores button state
bool oldTaskState = false; // stores old button state (time)
bool oldSnoozeState = false; // stores old button state (snooze)

// task
int task_pos = 0; // dial position
int time_pos = 0; // time dial position
int currentBuzz = -1;
// stores task position that is currently buzzing (-1 = nothing)
int timeMax = 24; // limit for dial turn (changes for hours/min/am-pm)
int timeButtonCount = 0; // hours/minutes/ampm
int lcdHour = 0; // for printing hour on LCD screen
int lcdMinute = 0;
int lcdTask = 0;
int rtcHour = 0;
int rtcMinute = 0;

// for each task (10 tasks)
int alarmHour[] = {12, 12, 12, 12, 12, 12, 12, 12, 12, 12};
int alarmMinute[] = {0, 0, 0, 0, 0, 0, 0, 0, 0, 0};
int alarmOn[] = {0, 0, 0, 0, 0, 0, 0, 0, 0, 0};

bool snoozeOn[] = {false, false, false, false, false, false, false, false, false, false};


void setup() {
  pinMode(MOTORPIN, OUTPUT);
  pinMode (TASK_BUTTON, INPUT_PULLUP);
  pinMode (TIME_BUTTON, INPUT_PULLUP);
  pinMode (SNOOZE_BUTTON, INPUT_PULLUP);
  pinMode(LEDPIN, OUTPUT);
  Serial.begin (9600);

  setSyncProvider(RTC.get);   // the function to get the time from the RTC

  screen.init();
  screen.backlight();
}

void loop() {
  //------------------------------------
  // Snooze
  //------------------------------------
  int snoozeState = digitalRead(SNOOZE_BUTTON); // task: enabled or not

  if (!snoozeState && oldSnoozeState) {
    screenLight = millis();
    Serial.println("SNOOZE");
    
    if (currentBuzz > -1) {
      if (alarmMinute[currentBuzz] < 45) {
        alarmMinute[currentBuzz] += 15;
         
      } else { 
        // alarmMinute[currentBuzz] >= 45
        alarmHour[currentBuzz] += 1;
        alarmMinute[currentBuzz] += (currentBuzz - 45);
      }
      
      snoozeOn[currentBuzz] = true;
      currentBuzz = -1;
    }
  }
  
  oldSnoozeState = snoozeState; // button thing

  //------------------------------------
  // Buzz Alarm
  //------------------------------------

  // loops through tasks
  for (int i = 0; i < 10; i++) {
    // checking if alarm needs to be activated
    
    if (hour() == alarmHour[i] && minute() == alarmMinute[i] && alarmOn[i] == 1) {
      screen.backlight();
      buzzing(); // pulses motor
      Serial.println("ALARM ON");
      currentBuzz = i; // save task position of current alarm going off
    }
  }
  
  if (currentBuzz == -1) {
    analogWrite(MOTORPIN, 0);
    digitalWrite(LEDPIN, LOW);
  }

  taskFunction();
  taskButton();


  //---------------------------------------
  // Time knob
  //---------------------------------------

  if (!(timeButtonCount % 3 == 0)) {
    
    int timeKnobMax = (timeMax) * 4;
    long newTimePosition;

    // account for 4 times extra readings
    if (timeKnob.read() > timeKnobMax) timeKnob.write(0);
    else if (timeKnob.read() < 0) timeKnob.write(timeKnobMax);

    newTimePosition = timeKnob.read();

    //TIME IS CHANGED
    if (newTimePosition != oldTimePosition) {
      screenLight = millis();
      oldTimePosition = newTimePosition;
      time_pos = oldTimePosition / 4;
      Serial.println(time_pos);
    }
  } else {
    timeKnob.write(0);
  }

  //------------------------------------
  // Time button
  //------------------------------------

  //Read time button
  int timeState = digitalRead(TIME_BUTTON);

  if (!timeState && oldTimeState) {
    screenLight = millis();
    screen.clear();
    Serial.print("LCD hour: ");
    Serial.println(lcdHour);
    timeKnob.write(0); // reset time knob position back to 0

    // display correct task and alarm time corresponding for that task
    if (task_pos < 9) {
      if (timeButtonCount % 3 == 1) alarmHour[task_pos] = time_pos;
      else if (timeButtonCount % 3 == 2) alarmMinute[task_pos] = time_pos;
    }
    // RESET time mode
    else if (task_pos == 9) { 
      if (timeButtonCount % 3 == 1) rtcHour = time_pos;
      else if (timeButtonCount % 3 == 2) rtcMinute = time_pos;
    }

    Serial.print(alarmHour[task_pos]);
    Serial.print(" : ");
    Serial.print(alarmMinute[task_pos]);
    Serial.print(" : ");
    Serial.println(alarmOn[task_pos]);

    timeButtonCount++;
    Serial.println(timeButtonCount);
    time_pos = 0;
    delay(300);
  }
  oldTimeState = timeState;
  
  // BUZZING
  if (currentBuzz > -1) {
    //display current task
    lcdHour = alarmHour[currentBuzz];
    lcdMinute = alarmMinute[currentBuzz];
    taskKnob.write(currentBuzz);
    task_pos = currentBuzz;
  }
  // set real time clock
  else {
    if (timeButtonCount % 3 == 0) {
      if (task_pos == 9) {
        lcdHour = rtcHour;
        lcdMinute = rtcMinute;
        
      } else {
        lcdHour = alarmHour[task_pos]; // display set hour on LCD
        lcdMinute = alarmMinute[task_pos];
        
      }
    }
    else if (timeButtonCount % 3 == 1) { //setting hour
      timeMax = 24;
      lcdHour = time_pos; //displays changing hour on LCD
      
      if (task_pos == 9) {
        lcdMinute = rtcMinute;
        
      } else {
        lcdMinute = alarmMinute[task_pos];
        
      }
    }
    else if (timeButtonCount % 3 == 2) { //setting minute
      timeMax = 60;
      lcdMinute = time_pos;
      
      if (task_pos == 9) {
        lcdHour = rtcHour;
        
      } else {
        lcdHour = alarmHour[task_pos];
        
      }
    }
  }


  screen.home();
  if (task_pos == 9) {
    screen.print("Reset Time");
  } else {
    screen.print("Task ");
    screen.print(task_pos + 1);
    if (snoozeOn[task_pos]) screen.print("*");
  }

  screen.setCursor(11, 0);
  if (hour() < 10) screen.print("0"); // 0 offset if < 10
  screen.print(hour());
  screen.setCursor(13, 0);
  screen.print(":");
  if (minute() < 10) screen.print("0"); // 0 offset if < 10
  screen.print(minute());

  screen.setCursor(0, 1);
  if (lcdHour < 10) screen.print("0"); // 0 offset if < 10
  screen.print(lcdHour);

  screen.setCursor(2, 1);
  screen.print(":");
  if (lcdMinute < 10) screen.print("0"); // 0 offset if < 10
  screen.print(lcdMinute);

  screen.setCursor(6, 1);

  if (alarmOn[task_pos] == 1) { // set alarm on
    // alarm on
    screen.setCursor(13, 1);
    if (task_pos == 9) {
      screen.print("SET");
    } else {
      screen.print("ON");
    }
  } else if (alarmOn[task_pos] == 0) { // set alarm off
    // alarm off
    screen.setCursor(13, 1);
    if (task_pos == 9) {
      screen.print("SET");
    } else {
      screen.print("OFF");
    }
  }

  //After a minute of inactivity, turn off the LCD backlight to save power
  if (screenLight + 60000 <= millis()) screen.noBacklight();
  else screen.backlight();
}


//pulses motor
void buzzing() {
  if (motorTimer + 1000 <= millis()) {
    motorState = !motorState; //flip state
    motorTimer = millis(); // reset timer
  }
  if (motorState) {
    analogWrite(MOTORPIN, 200);
    digitalWrite(LEDPIN, HIGH);
  }
  else {
    analogWrite(MOTORPIN, 0);
    digitalWrite(LEDPIN, LOW);
  }
}


//TASK knob
void taskFunction() {
  //Zachs method for looping the knob
  if (taskKnob.read() > 36) taskKnob.write(0);
  else if (taskKnob.read() < 0) taskKnob.write(36);
  long newTaskPosition = taskKnob.read();

  //TASK IS CHANGED
  if (newTaskPosition != oldTaskPosition) {
    oldTaskPosition = newTaskPosition;
    screenLight = millis();

    // reset time position and button count when task is changed
    timeButtonCount = 0;
    time_pos = 0;
    screen.clear();

    //print task position and time set for that task
    Serial.print("old task: ");
    Serial.print(oldTaskPosition);
    Serial.print(" / task_pos: ");
    Serial.println(task_pos);
    Serial.print(" / printing: ");
    Serial.println(newTaskPosition / 4);
  }

  task_pos = newTaskPosition / 4;
}


/*------------------------------------
  Task button
    if task knob is on position 11 -
    DISPLAY "RESET TIME" instead of "task 11"
    use time button to set current time
    hit task button to set
    when pressed, rtc.setTime(hours, minutes, seconds)
------------------------------------*/
void taskButton() {
  int taskState = digitalRead(TASK_BUTTON); // task: enabled or not

  if (!taskState && oldTaskState) {
    screenLight = millis();

    // RESET time on task 10
    if (task_pos == 9) {
      // set hour:min:sec
      tm.Hour = rtcHour;
      tm.Minute = rtcMinute;
      tm.Second = 0;
      // Apr 22 2019
      tm.Month = 4;
      tm.Day = 22;
      tm.Year = CalendarYrToTm(2019);
      if (RTC.write(tm)) {
        setSyncProvider(RTC.get);
      }
    }
    else { // regular alarm function
      if (snoozeOn[task_pos]) snoozeOn[task_pos] = false;
      
      // ***change 0 and 1 -> off and on for printing
      // setting the alarm: if alarm is off, turn it on
      if (alarmOn[task_pos] == 0 && currentBuzz == -1) {
        alarmOn[task_pos] = 1;
      }
      
      //if the alarm is set as on (and not buzzing), turn it off
      else if (alarmOn[task_pos] == 1 && currentBuzz == -1) alarmOn[task_pos] = 0;

      //if alarm is buzzing, we know alarm is on, so we turn it off
      if (currentBuzz > -1) {
        screen.clear();
        alarmOn[currentBuzz] = 0; // disable alarm
        analogWrite(MOTORPIN, 0); // turn off buzzing alarm
        digitalWrite(LEDPIN, LOW);
        Serial.println("ALARM OFF");
        currentBuzz = -1;
        taskKnob.write(0);
      }

      Serial.print("alarm on: ");
      Serial.println(alarmOn[task_pos]);

      // Alarm on
      if (alarmOn[task_pos] == 1) {
        screen.clear();
        screen.setCursor(13, 1);
        
        if (task_pos == 9) {
          screen.print("SET");
          
        } else {
          screen.print("ON");
          
        }
      } else if (alarmOn[task_pos] == 0) {
        // alarm off
        screen.clear();
        screen.setCursor(13, 1);
        
        if (task_pos == 9) {
          screen.print("SET");
          
        } else {
          screen.print("OFF");
          
        }
      }
      delay(300);
    }
  }
  oldTaskState = taskState; // button thing
}

This semester, we had the opportunity to work with a group of older people who attend OSHER Lifelong Learning Institute at Carnegie Mellon University. We collaborated with Lois, one of the OSHER students, and after an in-house interview where we discussed her needs and daily routine , we designed a volume regulator for her television. Often at night, when she watches TV with her husband Irv, the volume of the commercial is much louder than the volume of the regular TV program.  To solve this problem,  we decided to create a device that uses IR signals to interact with her TV.  When the TV gets too loud the device automatically turns down the volume. After 30 seconds, or by pressing a button, the device is able to turn the volume back up.

Link to:

In house interview

First Prototype

 Final Device

The device is designed to sit on the table in Lois’ den positioned directly in front of the television. The device will listen and if the sound that it hears is continuously above a threshold for a period of time, it will send IR codes to the TV to lower the volume.  After 30 seconds, the TV will send signals to raise the volume back up, presuming that the TV commercials have ended and the regularly scheduled TV program is back on.

Using the knob, Lois can change the threshold of the volume that she would like the TV to be set to. The button on top, is in some ways an override button. If for some reason the volume is lowered due to background noise, or the commercials end early, Lois can press the button to switch it to the other state. The device uses a 5V cable to connect to power and has a rocker switch to turn the device on/off.

Device as volume is being lowered: NeoPixel flashes green

Device as volume is being raised: NeoPixel flashes red

Device as volume threshold is set: NeoPixel illumination indicates the volume threshold as the knob is turned

Pushing the button on top to lower the volume, the NeoPixel flashes green to indicate that signals have been sent to the TV to lower the volume

Pushing the button on top to raise the volume, the NeoPixel flashes red to indicate that signals have been sent to the TV to raise volume again

Adjusting the knob on the device to change the threshold of the volume, NeoPixel adjusts to serve as a visual indication of what the threshold is.

Knob and Neopixel: The NeoPixel ring is a visual indication of what the device is doing: Lois wanted green to indicate that the volume is being lower, and red to indicate that volume is being raised. The white gives a visual representation of the volume—the knob is designed for ease of adjustment of volume.

Button: Button on top that can be pushed by Lois to change what signals the IR transmitter is sending

5V input: in order to allow Lois to use this in her den, we wanted to connect to a power outlet

Rocker Switch: Turns the device on/off without having to unplug the device

IR Transmitter: The clear acrylic is meant to allow the IR transmitter to send the signals, and since IR cannot send signals through opaque acrylic we decided to use clear.

Opened device: The wiring and the internal components of the device

Close up of internal electronics—notice the placement of the two breadboards: one is for IR transmitter and sound sensor while another is placed for the potentiometer knob above the Arduino

Narrative sketch of intended use

Process of Creating

Gantt Chart to help us organize our pathway leading up to the final critique

In terms of planning out our steps towards our final prototype, we were pretty on schedule with most of our plans. We started the actual building early on, so we were on schedule in that respect. We had some delays with the shipping of parts, and had some difficulty accurately sensing loudness. These drawbacks meant we didn’t finish a lot of our circuitry and code until the last four days. However, because some aspects of the physical final device (3D prints) did not depend on the finalization of the code and internal circuitry, we were able to continue moving forwards on all aspects of the project.

After the critique during the first prototype, we wanted to switch from an LCD screen to a more basic visual so that Lois wouldn’t need to get too close to the device to understand what it was doing–after brainstorming we determined that would be the NeoPixel would be a good alternative.

Talking with Lois during the first prototype critique to determine more specifically what her needs were (more info can be found on that previous documentation)

Working with the smaller NeoPixel and seeing the different effects it could have in the library (Look at the reflection on the laptop screen, pretty cool!)

At this point we had a lot of difficulty with the sound sensor we were given so after talking to Zach we changed it to the RobotDyn sound sensor

We began to work on the final physical model for Lois. We worked with understanding the dimensions of what we wanted our final product to be and how we wanted it to roughly look. Using the dimensions of the sensors and Neopixel that we already had, we created a cardboard representation of what the interactions associated with the device could be.

Lasercut cardboard templates to figure out dimensions of final product

After looking at the two cardboard mockups, we decided to work with the smaller model size—determining it was easier to print as well as more fitting in Lois’ den

After understanding dimensions we began to CAD out the final device.

Designing the final device on Fusion360 (2/3 parts are shown)

Designing the knob on Fusion 360

Failed 3D printed knob (the supports didn’t work on the Stratasys?)

We got the first and third print done, and as the second print was on the queue we explored the way that the sensors fit as well as the way that the NeoPixel shined through the PLA.

Working with the NeoPixel and figuring out the optimal placement for it within the device

Testing the fit of the rocker switch and 5v input

After all three pieces were printed, we realized that somewhere along the process of creating the 3D print, the joints where the pieces were meant to be connected did not connect properly. We had to dremel the joints down carefully; however, because the joints were hidden, it would not affect the overall craft.

Dremeling the parts to allow the pieces to fit together (dremeling Elena’s last brain cell away)

We decided to bondo the pieces of the 3D print that did not turn out smooth and because bondo has a color, we needed to spray paint two of the pieces. Due to availability of spray paint we only had gray.  After sanding down some rough edges, we added bondene to fragile parts of the print. We were now ready to add the electrical components into the device.

Bondo to smooth out rough parts of the print

Sanding down the bondo to make the edges and print smoother

Spray painted part

Bondene’d the weaker parts of the print (especially the NeoPixel print area to ensure that the part would not crack

As the physical model was being made, we worked on the Neopixel functionality as well as the sound sensor detection.

Working with the Neopixel and combining it with the code for the sound detector

Even after switching the sound sensor, when the sensor picked up on any sound, the reading would spike, which meant that if someone made a loud noise on the television program, the volume would automatically be lowered. We didn’t want this to happen so after talking to Zach, we realized that we could average out the sound readings for a period of time  and that way if the sound was continuously loud for about two-three seconds, the television volume will be lowered. 

After figuring out the code we began to work through the wiring diagram and soldering the pieces onto the Arduino.

Collaborative circuitry diagram creation!

Final Circuitry Diagram drawn on white board

As we began to house the electronics in the device, we had some difficulty with the potentiometer placement so that the knob would spin on an axis. Therefore, we had to create a second level of sorts within the device using a separate breadboard from the one where the IR sensors were located. 

Creating a second level to hold the breadboard for the potentiometer in place

Final placement of internal components

Finished product before critique with electronics within the housing after testing with the projector.

After the final prototype was assembled, we tested out functionality with the projector volume and simulated the television volume with our voice. Then it was on to the final critique!

Conclusion and lessons learned

We received a lot of comments to the effect of “could add more control on this box.” Some people suggested having other dials to control the commercial timer. At the moment, it is permanently set to 30 seconds, but some programs have commercials that are a minute long or more. In addition, people suggested making a dial that changes how much the volume is decreased. Both of these dials would be easily implemented in the Arduino code. However, they would have required a good amount of more testing. In the future these could be a possibility.

On the other hand, the vast majority of our comments praised our “clean fabrication,” but a few comments brought attention to our acrylic window saying, “clear window looks a bit unpolished.” The functionality of the acrylic was to make sure that the infrared LED would transmit the cleanest signal to the TV, however, by doing so we exposed the inner wires and electronics. I do believe we could make a future design that makes sure the electronics are not as visible. In particular, most remotes have a nice form around their infrared LED that seems to direct the signal better. Perhaps we would benefit from implementing a housing for the LED that imitates that shape.

Other comments involved the fact that the box had to face the TV and that the “Red vs. green lights confuse me.” However, these decisions were made under Lois’ direction. The way she sits as she watches TV makes it so that it is okay for the box to face the TV. In addition, she made it clear that she would like the volume-down phase to flash green and the volume-up to flash red.

All in all, there were few things that Lois insisted on.  A challenge we faced was that after our first meeting with Lois we did not have a clear direction or path.

In the future, we would have a more impactful interview, especially if we emphasized to her that we would like specific ideas. What we should have noticed in the first interview was just how interested Lois was in finding a solution to her TV loudness. It was surprising how excited and pleased Lois was at every step of the way in our prototypes. After our final critique, she brought up the idea of marketing our product, which was a bit of a foreign idea for us However, we felt is was one of the many benefits of working with an older person because their perspective is much broader and more experienced than ours. It was truly a pleasure to work with her.

Technical Details

Schematic

Schematic for TV volume regulator

Code

/*
 * TV Volume Regulator
 * Elena Deng, Connor Maggio, Aditi Raghavan
 * 
 * Device Functionality:
 * If the volume coming from the TV is too loud,
 * the device sends an IR remote codes to the TV to
 * turn down the volume. After 30 seconds the 
 * device sends IR codes to turn the volume back up. 
 * A momentary pushbutton can also be used to turn the
 * volume up or down. 
 * 
 * Required Components: Arduino Uno, Potentiometer, IR LED, 
 * IR receiver, Momentary Push Button, Toggle Switch, 
 * Power Source, Audio Sensor (RobotDYN Sound Sensor), 
 * 24 LED Neopixel, 5V Power Source 
 * 
 * Inputs: Button, Sound Sensor, Potentiometer
 * Outputs: IR signals, Neopixel 
 * 
 * PINS:
 * PIN 3 - IR LED
 * PIN 6 - Neopixel
 * PIN 8 - Momentary Button
 * PIN 11 - IR Receiver
 * PIN A1 - Sound sensor
 * PIN A2 - Potentiometer
 * 
 * Notes: 
 * 1. Neopixel library requires that Pin 6 is used. 
 * 2. The IR libaries require the IRLED to be pin 3
 * 3. IR Codes are specific to one TV and should be changed 
 * accordingly
 * 4. Connect 5V power source to Vin on the Arduino Uno
 *
 */

#include <ir_Lego_PF_BitStreamEncoder.h>
#include <boarddefs.h>
#include <IRremoteInt.h>
#include <IRremote.h>
#include <Wire.h>
#include <Adafruit_NeoPixel.h>
#define LED_PIN     6

// LED on the Neopixel
#define LED_COUNT  24

// NeoPixel brightness
#define BRIGHTNESS 50

// Declare our NeoPixel strip object:
Adafruit_NeoPixel strip(LED_COUNT, LED_PIN, NEO_GRBW + NEO_KHZ800);

// Declaring pins 
const int BUTTON = 8;
const int AUDIOPIN = A1;
const int POTPIN = A2;
const int IR_TX = 3;
const int IR_RECV = 11;\

// Declaring thresholds
int ledThresh;
int oldLedThresh;
int potVal;
int audioThresh;
int threshold;

// Setting up ir receiver and sender
IRrecv irrecv(IR_RECV);
IRsend irsend;
decode_results results;

// Averaging
int dB_samples[200];
int count = 0;
int current_dB;
unsigned long sum = 0;
unsigned long average;

bool remoteButtonPressed;
bool turnVolumeDown = false;

//Detecting Audio
bool peakWait = true;
bool nonPeakWait = false;
unsigned long peakTimer;
unsigned long belowThres = 2000;
unsigned long nonPeakTimer;
unsigned long aboveThres = 0;

//Button
int buttonState = HIGH;
int oldButtonState = HIGH;
bool buttonPressed;

//Commercial data
unsigned long cmlStart;
unsigned long cmlTimer;

//IR Values
String remoteButton = "1897e817";
int muteButton;
int unmuteButton;

// FSM States
const int MODE_LISTEN = 0;
const int MODE_MUTE = 1;
int currentMode = MODE_LISTEN;

struct TV{
  int  volThreshold = 100;
  bool isMute = 0;
} LoisTV;

void setup() {
  pinMode(BUTTON, INPUT_PULLUP);
  pinMode(AUDIOPIN, INPUT);
  pinMode(POTPIN, INPUT_PULLUP);
  pinMode(IR_RECV, INPUT);
  pinMode(IR_TX, OUTPUT);
  Serial.begin(9600);
  irrecv.enableIRIn();
  irrecv.blink13(true);
  strip.begin();
  strip.show();
  strip.setBrightness(50);
}

void loop() {
  /*
   * Potentiometer Threshold meter
   */
  potVal = analogRead(POTPIN);
  audioThresh = map(potVal, 0, 1023, 0, 500);
  int temp = map(potVal, 0, 1023, 0, 48);
  ledThresh = map(temp, 0, 48, 0, 24);
  if (ledThresh != oldLedThresh){
    ledPotentiometer(ledThresh);
    oldLedThresh = ledThresh;
  }
  LoisTV.volThreshold = audioThresh;

  /*
   * Averaging
   */
  current_dB = analogRead(AUDIOPIN);
  sum = sum + current_dB;
  sum = sum - dB_samples[count];
  dB_samples[count] = current_dB;
  count = (count+1) % 200;
  int average = sum/200;

  /*
   * Button
   */
   buttonState = digitalRead(BUTTON);
   if (!buttonState && oldButtonState) buttonPressed = true;
   oldButtonState = buttonState;

  /*
   * Detecting Loudness
   */
  if ((average >= LoisTV.volThreshold) && peakWait) {
    peakTimer = millis();
    nonPeakWait = true;
    peakWait = false;
    belowThres = millis() - nonPeakTimer;
  }
  if ((average < LoisTV.volThreshold) && nonPeakWait) {
    nonPeakTimer = millis();
    peakWait = true;
    nonPeakWait = false;
    aboveThres = millis() - peakTimer;
  }
  

  /*
   * Mode Management and Functionality
   */
  switch(currentMode)
  {
    case MODE_LISTEN:
      if ((aboveThres >= 1000) 
          || ((aboveThres >= 200) && (belowThres < 400))) {
        turnVolumeDown = true;
      }
      /*If above threshold or the button is pressed 
       when in listening mode, send turn volume down codes
      */
      if (turnVolumeDown || buttonPressed)
      {
        cmlStart = millis();
        pulseGreen(5);
        irsend.sendNEC(0x1897926D, 32); // projector Volume Down
        delay(100);
        irsend.sendNEC(0x1897926D, 32); // projector Volume Down
        delay(100);
        irsend.sendNEC(0x1897926D, 32); // projector Volume Down
        delay(100);
        irsend.sendNEC(0x1897926D, 32); // projector Volume Down
        delay(100);
        irsend.sendNEC(0x1897926D, 32); // projector Volume Down
        delay(100);
        irsend.sendNEC(0x1897926D, 32); // projector Volume Down
        delay(100);
        irsend.sendNEC(0x1897926D, 32); // projector Volume Down
        delay(100);
        irsend.sendNEC(0x1897926D, 32); // projector Volume Down
        delay(100);
        LoisTV.isMute = true;
        buttonPressed = false;
        currentMode = MODE_MUTE;
      }
      break;
    case MODE_MUTE:
      cmlTimer = millis() - cmlStart;
      /* Decodes results from the TV remote to see if certain button
       *  is being pressed.
       */
      if (irrecv.decode(&results)) {
        Serial.println("sent");
        Serial.println(String(results.value, HEX));
        Serial.println(String(results.value, HEX));
        Serial.println(String(results.value, HEX));
        if (String(results.value, HEX) == remoteButton) {
          remoteButtonPressed = true;
        }
        irrecv.resume();
      }
      
      /* If the button is pressed or the center button on the TV remote is pressed
       *  or the timer has elapsed 3000 seconds, the device sends codes to turn
       *  the volume back up on the TV
       */
      if (buttonPressed || remoteButtonPressed || (cmlTimer >= 30000))
      {
        pulseRed(5);
        irsend.sendNEC(0x189712ED, 32); // projector Volume Up
        delay(40);
        irsend.sendNEC(0x189712ED, 32); // projector Volume Up
        delay(40);
        irsend.sendNEC(0x189712ED, 32); // projector Volume Up
        delay(40);
        irsend.sendNEC(0x189712ED, 32); // projector Volume Up
        delay(40);
        irsend.sendNEC(0x189712ED, 32); // projector Volume Up
        delay(40);
        irsend.sendNEC(0x189712ED, 32); // projector Volume Up
        delay(40);
        irsend.sendNEC(0x189712ED, 32); // projector Volume Up
        delay(40);
        irsend.sendNEC(0x189712ED, 32); // projector Volume Up
        delay(40);
        LoisTV.isMute = false;
        buttonPressed = false;
        aboveThres = 0;
        belowThres = 2000;
        turnVolumeDown = false;
        currentMode = MODE_LISTEN;
      }
      break;
  }
}

// Pulses Neopixel Red
void pulseRed(uint8_t wait) {
  for (int j = 0; j < 256; j++) { // Ramp up from 0 to 255
    // Fill entire strip with white at gamma-corrected brightness level 'j':
    strip.fill(strip.Color(strip.gamma8(j), 0, 0, 0));
    strip.show();
    delay(wait);
  }

  for (int j = 255; j >= 0; j--) { // Ramp down from 255 to 0
    strip.fill(strip.Color(strip.gamma8(j), 0, 0, 0));
    strip.show();
    delay(wait);
  }
}

// Pulses Neopixel Green
void pulseGreen(uint8_t wait) {
  for (int j = 0; j < 256; j++) { // Ramp up from 0 to 255
    // Fill entire strip with white at gamma-corrected brightness level 'j':
    strip.fill(strip.Color(0, strip.gamma8(j), 0, 0));
    strip.show();
    delay(wait);
  }

  for (int j = 255; j >= 0; j--) { // Ramp down from 255 to 0
    strip.fill(strip.Color(0, strip.gamma8(j), 0, 0));
    strip.show();
    delay(wait);
  }
}

// Maps potentiometer values to Neopixel LED's
void ledPotentiometer(uint8_t numLed){
  for (int i = 0; i < 24; i++) {
    if (i < numLed+1) { //the +1 keeps one LED on at all times
      strip.setPixelColor(i, 180, 180, 180); // turn on this LED, also adjusts colors                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       ); //turn on this LED, adjust values to affect colors and brightness of LEDs
     
    }
    else {
     strip.setPixelColor(i, 0, 0, 0); //turn off this LED
     
    }
    strip.show();
  }
  
}

The Intro to Physical Computing final project is to build a device that will help an older friend in way that is unique and specific to them; for us that person was Peter. As he lives in the penthouse of his apartment, the wind gusts on his balcony are really strong. This has caused him to have to tie down all the furniture out there as well as make sure that it is safe enough to go out there. When he babysits his granddaughter, she isn’t allowed on the balcony because it may be unsafe for her. Because of this, we decided to build him a personalized display of the wind speeds on his balcony. For more information about our meeting with him, check more on our process wordpress, and read on to learn more about our final work.

What We Built 

Our project measures the speed of the wind on his balcony and then displays this information in three different ways. The max speed is shown with a speedometer, the current speed is shown on a digital display, and the max wind speeds of the past 7-70 minutes are shown in a bar/line graph based on the setting Peter sets it as. This is all housed on a stand in a compact wooden box with a cloud and the 3 displays on the front with the switches on the right side panel.

Detail 1

Detail 2

Detail 3 Numbers

Detail 4 Side

After a windy day, Peter arrives home wondering just how fast it was so high up, so he heads over to Cool Cloud to see the max speed and the history throughout the last hour or so. He’s not surprised that it was 72+mph, but is happy that his suspicions were confirmed that it truly did get that fast on his balcony.

On a nice summer day, Peter wants to have a barbeque on his balcony. However, the news said that there would be mild winds, which might have meant anything up so high in the past. Now though, he knows what the wind is like on such days because of our Cool Cloud. He now knows that it will be perfectly safe and can now better plan future events outside based off what the weather says and the past speeds on such days. He can also finally let his granddaughter see the amazing view his apartment offers without fear of the wind.

How We Got Here

We started off by picking an off-the-shelf sensor that could detect wind speed. The anemometer we chose was a highly durable and weatherproof model that could measure up to 72 mph winds, which seemed ideal for our purposes.

The anemometer

Next, we started prototyping the indoor portion, figuring out what kind of visualizations we would have and how we would package them. We wanted to focus on the aesthetics of the indoor module, as that would set this project apart from a generic weather sensor.

Lasercut wooden grid for the LED bar graph

Detail shot of prototype LED strip

Functional prototype with a potentiometer dummy input

Showing our prototype to Peter

The biggest challenges at this point were all design choices. The scope of the project was pretty wide open, and we had to decide how we were going to present the information to Peter in a useful and appealing way. We spent a lot of time sketching, CADing, and prototyping different approaches.

For the final version, our challenges were to package everything nicely in a box and to weatherproof the outdoor module. This meant figuring out a secure way to mount the anemometer to Peter’s railing as well as building an enclosure that could keep the electronics from getting wet.

To secure the anemometer, we designed a 3-part clamping system that could bolt together on a T-joint of Peter’s railing.

Brainstorming a clamping system

Making the side panels

Clamp prototype

Creating the mounting box to be 3D printed

We 3D printed a final version of the clamping pieces. For the indoor portion, we still had to build our box and solder all of our electronics.

Lasercut pieces for the enclosure

Making the stand

Back panel with some soldered components

All components soldered and mounted

The toughest part of the final was the small last-minute adjustments we had to make to get everything to work smoothly. For instance, we had to cut some of our lasercut pieces to size because they were off by a little bit. We also added a hole to be able to access the Arduino’s USB port. It also took us a few tries to make the standoffs that the servo and LED matrices are mounted to. Technically, none of the components in this project were that challenging to use, and we were confident we could achieve full functionality. Integrating everything seamlessly and mechanically fabricating the whole thing was challenging.

Conclusions and Lessons Learned

The crit made us feel pretty good about our project as most people said it was pretty cool and worked well. One question we got was as to why our graph read from left to right with the left being the most recent instead of the other way around. This is interesting because we thought this was most logical as you read a book from left to right. It was brought up that in most bar graphs the most recent information is on the right and even for a book, the most recent information is technically on the right. This is a pretty sound argument that we never thought of and would have been a pretty quick fix if we chose to do so. One criticism we received from the written feedback is that our indoor component could have looked better, more specifically as two people pointed out that it may have looked better if we used “a different material than wood” because “you can see the burn marks.”  We liked the contrast the wood provided with the frosted acrylic and thought it would have a nice aesthetic on a table or wall; however, we could have chosen another material like a different color acrylic that would have had contrast without burn marks. Another concern was that “it will not survive PGH winter.” This is a concern that we also have about our outdoor component, but the anemometer is said to be able to take the weather and the screws should be able to hold up. One feature that Peter mentioned would have been nice is if he could plug his  computer in and see the history of it over its entire time running or if data could have also been transmitted to his phone. These sound like pretty interesting features, that we may have been able to include had we been informed of the desire earlier as well as if we had more time on the project. Other than that Peter was happy with our cloud display and really liked that we decided to include the toggle switch for him to go between a bar and line graph.

This was a really refreshing and fulfilling experience because we got to build something that we know will be used and is helpful to someone other than ourselves or simply to meet an assigned task. It was surprising how long it took us to figure out that this project is what Peter clearly wanted from the very beginning. Talking to Peter throughout the designing and building process was also a lot more seamless than originally anticipated. We do however think it would also be a good idea to ask if there were any additional features that he wanted us to include while making the final product (after the prototype meeting). This way we could discuss with him the feasibility of such things and maybe incorporate an aspect of it. Overall we are really happy with how our project turned out in that it worked exactly as we meant it too.

While we are happy with our project, there are still some things that we would have done differently if we were making it again. Hinging the back of the box so that it can be opened back up in case something somehow went wrong with the wiring last second is something to keep in mind for all our future projects and would have been very helpful for our peace of mind. Additionally, it would be a good idea to test everything together earlier in case there wasn’t as simple of a fix as we found and it were to take longer to figure it out.

Technical Details

/*
 * Cool Cloud
 * 
 * Alton Olson, Vicky Zhou, Seema Kamath
 * 
 * This is the transmitter that lives outside and sends
 * wind data to the receiver indoors.
 * 
 * Pins:
 * A0 - anemometer output
 * 7 - radio CE
 * 8 - radio CSN
 */

#include <SPI.h>
#include <nRF24L01.h>
#include <RF24.h>

RF24 radio(7, 8); // CE, CSN
const byte address[6] = "00001";

void setup() {
  // set up radio
  radio.begin();
  radio.openWritingPipe(address);
  radio.setPALevel(RF24_PA_HIGH);
  radio.stopListening();
}

void loop() {
  // read anemometer value
  int reading = analogRead(A0);
  // send anemometer value
  radio.write(&reading, sizeof(reading));
  delay(1000);
}

/*
 * Cool Cloud
 * 
 * Alton Olson, Vicky Zhou, Seema Kamath
 * 
 * This is the receiver that lives indoors and displays
 * all the visualizations.
 * 
 * Pins:
 * A0 - slider pot for adjusting graph interval
 * 3 - switch for line graph/bar graph
 * 7 - radio CE
 * 8 - radio CSN
 * 9 - servo
 * 10 - LED strip
 */

#include <PololuLedStrip.h>
#include <Wire.h>
#include <Adafruit_GFX.h>
#include "Adafruit_LEDBackpack.h"
#include <SPI.h>
#include <nRF24L01.h>
#include <RF24.h>
#include <Adafruit_TiCoServo.h>

const int LED_COUNT = 49;
const int SERVO_PIN = 9;
const int LED_PIN = 10;
const int POT_PIN = A0;
const int SWITCH_PIN = 3;
const int RADIO_CE_PIN = 7;
const int RADIO_CSN_PIN = 8;
const int TENS_MATRIX_ADDR = 0x70;
const int ONES_MATRIX_ADDR = 0x71;
const int GRID_WIDTH = 7;
const int GRID_HEIGHT = 7;
const int DAY_MILLIS = 86400000;

// binary data for the 8x8 matrix images
const uint64_t TENS_IMAGES[] = {
  0x0000000000000000,
  0x7020202020203020,
  0xf810204080808870,
  0x7088808060808870,
  0x8080f88890a0c080,
  0x70888080780808f8,
  0x7088888878088870,
  0x10101020408080f8,
  0x7088888870888870,
  0x708880f088888870
};
const uint64_t ONES_IMAGES[] = {
  0x0e1111111111110e,
  0x0e04040404040604,
  0x1f0204081010110e,
  0x0e1110100c10110e,
  0x10101f1112141810,
  0x0e1110100f01011f,
  0x0e1111110f01110e,
  0x020202040810101f,
  0x0e1111110e11110e,
  0x0e11101e1111110e
};

// display one of the binary coded images
void displayImage(Adafruit_8x8matrix matrix, uint64_t image) {
  matrix.clear();
  for (int i = 0; i < 8; i++) {
    byte row = (image >> i * 8) & 0xFF;
    for (int j = 0; j < 8; j++) {
      matrix.drawPixel(i, 7 - j, bitRead(row, j));
    }
  }
  matrix.writeDisplay();
}

// convert from (x, y) to an LED index on the strip
int getLEDnum(int x, int y) {
  if (y % 2 == 0) {
    return (6 - x) + 7 * y;
  }
  else {
    return x + 7 * y;
  }
}

// convert from a speed to a bar graph height
int getYval(int currentSpeed) {
  if (currentSpeed <= 5) {
    return 0;
  } else if (currentSpeed <= 15) {
    return 1;
  } else if (currentSpeed <= 25) {
    return 2;
  } else if (currentSpeed <= 35) {
    return 3;
  } else if (currentSpeed <= 45) {
    return 4;
  } else if (currentSpeed <= 55) {
    return 5;
  } else {
    return 6;
  }
}

// color coding (red, yellow, green)
rgb_color getColor(int y) {
  if (y == 6) {
    return rgb_color(255,0,0);
  } else if (y == 5) {
    return rgb_color(255,0,0);
  } else if (y == 4) {
    return rgb_color(255,255,0);
  } else if (y == 3) {
    return rgb_color(255,255,0);
  } else if (y == 2) {
    return rgb_color(255,255,0);
  } else if (y == 1) {
    return rgb_color(0,255,0);
  } else {
    return rgb_color(0,255,0);
  }
}

// global variables
RF24 radio(RADIO_CE_PIN, RADIO_CSN_PIN);
const byte address[6] = "00001";

Adafruit_TiCoServo servo;
PololuLedStrip<LED_PIN> ledStrip;
rgb_color colors[LED_COUNT];

Adafruit_8x8matrix tensMatrix = Adafruit_8x8matrix();
Adafruit_8x8matrix onesMatrix = Adafruit_8x8matrix();

// history for bar graph
int maxRecentVals[GRID_WIDTH];
// toggle for line graph/bar graph
bool isLineGraph = false;
// interval to shift bar graph
unsigned long graphShiftInterval = 1000; // in ms
// timer for bar graph shift
unsigned long loopStartTime;
// variables to track peak speed
int currentSpeed, peakcurrentSpeed;
unsigned long peakcurrentSpeedTime;
// servo angle
int motorAngle = 0;

void setup() {
  Serial.begin(9600);
  pinMode(POT_PIN, INPUT);
  pinMode(SWITCH_PIN, INPUT_PULLUP);
  servo.attach(SERVO_PIN);

  tensMatrix.begin(TENS_MATRIX_ADDR);
  onesMatrix.begin(ONES_MATRIX_ADDR);

  loopStartTime = millis();

  radio.begin();
  radio.openReadingPipe(0, address);
  radio.setPALevel(RF24_PA_HIGH);
  radio.startListening();

  for (int i = 0; i < GRID_WIDTH; i++) maxRecentVals[i] = 0;
}

void loop() {
  // get radio input from transmitter
  if (radio.available()) {
    int anemometerVal = 77;
    radio.read(&anemometerVal, sizeof(anemometerVal));
    // convert raw analog value to wind speed in mph
    currentSpeed = map(anemometerVal, 77, 412, 0, 72);
  }
  // track daily peak speed
  if (currentSpeed > peakcurrentSpeed || millis() - peakcurrentSpeedTime > DAY_MILLIS) {
    peakcurrentSpeed = currentSpeed;
    peakcurrentSpeedTime = millis();
  }
  // get graph shift interval
  int potVal = analogRead(POT_PIN);
  // map to 1 minute - 10 minute range
  unsigned long graphShiftInterval = map(potVal, 0, 1023, 60000, 600000);

  int prevMotorAngle = motorAngle;
  // convert speed to angle
  motorAngle = map(peakcurrentSpeed, 0, 70, 15, 180);
  // only write to servo on angle change
  if (motorAngle != prevMotorAngle) servo.write(motorAngle);

  // shift graph
  if (millis() - loopStartTime > graphShiftInterval) {
    loopStartTime = millis();
    for (int i = GRID_WIDTH - 1; i > 0; i--) {
      maxRecentVals[i] = maxRecentVals[i - 1];
    }
    maxRecentVals[0] = 0;
  }
  // continuously update newest graph column
  maxRecentVals[0] = max(currentSpeed, maxRecentVals[0]);

  // clear LED strip
  for (int i = 0; i < GRID_WIDTH * GRID_HEIGHT; i++)
    colors[i] = rgb_color(0, 0, 0);

  // display bar graph
  isLineGraph = digitalRead(SWITCH_PIN);
  for (int x = 0; x < GRID_WIDTH; x++) {
    int val = maxRecentVals[x];
    if (isLineGraph) {
      int y = getYval(maxRecentVals[x]);
      colors[getLEDnum(x,y)] = getColor(y);
    } else {
      for (int y = 0; y <= getYval(val); y++) {
        colors[getLEDnum(x,y)] = getColor(y);
      }
    }
  }
  ledStrip.write(colors, LED_COUNT);

  // display instantaneous reading on 8x8 LEDs
  int tensIndex = (currentSpeed / 10) % 10;
  int onesIndex = currentSpeed % 10;
  displayImage(tensMatrix, TENS_IMAGES[tensIndex]);
  displayImage(onesMatrix, ONES_IMAGES[onesIndex]);
}

design files

The open box with features the the locking mechanism servo towards the middle and the groove at the top.

The closed package-box which has a sign for the delivery drivers “For deliveries. Lift Lid. Place Box.”

Brief Summary of work:

Our names are George, Yael and Enock, and for our final project we collaborated with Emily, a Carnegie Mellon University OSHER student, to try and improve a facet of her life. Though our collaborative discussions produced several promising ideas and directions, we eventually settled on attempting to solve Emily’s package theft problem. We learned through our in person meetings with her that she not only lives in a corner street house, but that her patio is fully exposed to potential package stealing crimes of opportunity. These issues were further exacerbated by school bus stops across the street from her house, making her area highly trafficked during prime package delivery times. In fact, she reflected more than once about children walking home from the bus and stopping at her property to steal packages. We refer the interested reader here for additional information and motivation behind our choosing this project. Soon after meeting with Emily, we created our initial prototype device, which centered around having a ‘smart lock’ that could recognize when a package has been placed and lock accordingly. We then had the opportunity to not only demo it to our peers and her, but also receive valuable feedback about the best ways to improve our project so that it may better Emily’s life. A thorough discussion of our prototype and critique can be found here, while below we present our finial documentation for the final version of our package thief prevention box.

What We Built:

We designed a large lofted and raised  weather resistant wooden anti-theft box designed to sit on a porch and prevent packages from being stolen. At the core of our box is a pressure sensor which serves to detect whether or not packages have been placed inside. Once a package has been placed, the device automatically locks and waits until its owner unlocks it with a key. In its unlocked state, the owner is able to not only collect their package(s) from inside, but can also recalibrate the package sensing in case it is no longer accurately detecting packages. Once the package has been taken out and/or the sensing has been recalibrated, the owner turns the key to set the device back into the ‘package detection’ state. The device is powered by four batteries found in a cubby below its floating base, and further doubles as a surface to put items on, allowing the owner to place bags on it while opening the front door.

Shown above is a 2-dimensional version of the information given to the CNC mill

Narrative Sketch:

Emily is out of the house when she receives a text message from her son. He tells her that Amazon has notified him of a package delivery. She tells him not to worry, the new package collector on the porch should handle the job well.

Back home the package delivery man steps up onto Emily’s porch and places her packages into our box, which is labeled “For Deliveries: Lift Lid. Place Box” A few seconds after the package is placed and the box’s lid is closed,  a mechanical “whirr” can be heard. A few hours later a school bus stops at her street corner. School student David walks off the bus and spots the new package collection box. He looks around before trying to pry it open, hoping to be rewarded with a package. Fortunately, the box resists his efforts and does not open. Defeated, David tries one last resort: picking up the box entirely and carrying it home. However he is unable to as the package collection bin is nailed to the porch floor. Already being on Emily’s property longer than he wishes, and being too afraid of getting caught continuing to poke around, David quickly leaves.

Later that day, Emily returns home and uses her key to unlock the box. She safely removes her package from within and turns the key in the lock to enable it to detect future packages.

Placing a package into the box

Auto-locking feature of the box.

How We Got Here:

Staying on Track:

Our initial plan was:

Process Documentation: On all future class meetings, we will take photos, as well as the last few days.

Team Meeting: Minimum of all class days, likely on weekends

Idea Finalization: 4/15-4/17

Mechanical Planning: 4/10-4/12

Electronics Planning: 4/10-4/12

Materials Ordering: 4/10-4/14

Mechanical Fabrication: 4/15-4/23, bring to class 4/24

Electronics Fabrication: 4/17-4/22

Software Development: 4/17-4/24

Integration and Testing: 4/24-4/26

Repair/Redesign: 4/26-4/28

Presentation Prep: 4/28

Throughout the project we were mostly able to stick to our proposed plan, predominantly because we recognized the potential for things going wrong and taking much longer than expected. Thus we wanted leave ample time to confront any potential issues. However, as we started facing setbacks, we strayed from the plan as we had to merge different sections together. For instance, though we had the full structure of the box ready by April 24th, when our sensors broke (as discussed below), we had “go back” and add more structure to the box for the last working pressure sensor. This resulted in merging the repair, construction, and integration tasks together to resolve our issue. 

Below we cover and reflect on our creation process from after the prototype,  as our previous documentation exhibits thorough discussions on prior milestone work.

Anti-Theft Box Structure Design

Initially the mechanical fabrication seemed like a daunting task. To construct the box, we used 8′ x 4′ sheets of Baltic Birch, 3/4″ thick. The task of cutting every piece out quickly and accurately enough to make a box in the given time seemed nearly impossible. The size and thickness of the box would make typical wood shop fabrication very difficult, it would require many jigs and extra care, especially for the U-shaped cutout for the legs. So rather than cutting by hand, a CNC mill was used. The CNC mill was given a 3D model in Rhino 6 which it referenced as it cut. The entire CNC milling process took about 6 hours. From there, the pieces were pulled in a cart to the architecture wood shop. The next few days were spent sanding the faces and edges of every piece before giving it a coat of water-based polyurethane. After each coat, the wood would set for a few hours before the other side could also receive the coat. This was done 2 or 3 times for each of the 27 pieces. From there, the box was transported to the physical computing lab and hinges were attached. At first we tried only one set of hinges but found that 2 were required or else the lid rocks. After some redesigning of electrical component, a few more laser cut pieces of wood were glued inside the box to include new ideas and act as housing for various electronics.

The CNC Mill cutting out the sides of the box

Each individual part being moved from the CNC Mill to the workshop for sanding, coating, and construction

The completed box and base before they were screwed together

Yael screwing in the hinges to the box

Hardware Component Integration

One of the most interesting and challenging aspects of this project was designing our box’s hardware integration and configuration to enable it to function properly. In this section we cover three chief challenges we faced in this creation component: (i) the pressure sensing configuration, (ii) the locking mechanism, and (iii) our device powering.

Pressure Sensor Configuration

Our first challenge involved configuring our box’s pressure sensors to accurately detect package placement. Our initial prototype design utilized a single small rounded pressure sensor which was fully exposed on the base of the prototype box. As a result it could only detect packages that were placed directly over it, which was highly suboptimal for our project as we needed a reliable method to detect packages which were placed anywhere in the box (not necessarily over our sensors). Thus, we proposed a new solution as illustrated in the image below. We planned to elevate four sensors in each corner of the box (allowing us to hide the Arduino in the space below the sensors and the box’s base), and place a wooden cover which stretched the area of the box over them. This way, when a package was placed in the box, the cover would distribute pressure readings to each sensor, regardless of its location, thereby activating each.

Our initial plans for pressure sensor locations in our anti-theft device. Notice the square in the center of the picture with ‘A’ and ‘P’s. The boxes with symbols ‘P’ denote pressure sensor locations, while the box with ‘A’ represents the ‘Arduino’. The blue lines emanating from the Arduino denote connection wires.

We next turned our attention to preparing the sensors for device placement by soldering wires to their wire strips. This was a challenging because their positive and negative wires were very close to one another making it very difficult to solder one end without accidentally causing a short. In order to judge our soldering success and the integrity of each resultant component, we built an example circuit which exclusively tested whether or not the sensor still worked after soldering. This allowed us to determine any potential problems with our soldering early on, before integrating each to the device.

After successfully soldering each sensor, we set about attaching them in their respective regions around the box. While there were many different ways to do this, we ultimately narrowed down our decision between tape and glue. We believed glue might be advantageous as it is much more durable, and would thus decrease the likelihood that the sensors would deviate from their initial placements. However, we were very concerned with any negative effect glue might have on the sensor’s fragile sensor wire strips and its bottom. Thus, we decided to instead use tape, which at the cost of durability, would be more flexible and thus hopefully not break the sensors. Unfortunately, we quickly found this not to be the case as four of our five sensors immediately broke after taping them to the box. Under further examination we found the underlying cause to be our bending the wire strips to snugly attach the sensors to box: bending the strips caused them break away from the main sensing component of the pressure sensor, rendering it useless. This was a significant setback, as we were now in the same position as we were in our prototype (a single sensor to detect packages anywhere). However, we persevered and quickly found a new solution: place the remaining working sensor on an elevated compartment in the center of the box, and then rest the wooden cover over it. In theory, this solution would still be effective as the cover would still react to packages being placed anywhere, which would in turn affect the sensor. We quickly built a make shift elevated compartment for the sensor, and carefully taped it to the compartment (making sure to avoid touching the wire strips). To our satisfaction, the sensor worked exactly as we had hoped. 

Locking Mechanism

The second major challenge of our device creation was designing an effective locking mechanism. This could be broken down into two components: (i) the lock itself and (ii) the way in which to (un)lock it. In this section, we thoroughly discuss our approaches to each part.

Overhead shot of the package in the box, the servo wont lock since lid isn’t closed.

The Lock

The most difficult component of the lock task was designing the lock itself. In our initial prototype, we opted for motorized latch type lock, where a hobby servo motor attached to the front of the box would rotate an L-shaped latch into a slot on the lid to lock or unlock the box. However, our approach was very simple and we did not account for any weak points of contact (tugging on the box’s lid would easily break the lock). Thus, for our final product, we experimented with two different approaches: (i) using a solenoid lock (similar to a door lock), and (ii) re-designing the servo motor lock to more capably handle break-in attempts.

We were first drawn to solenoids following discussions with our professor and TA, who suggested them as a very robust and secure way to lock objects. This primarily stemmed from their metal exteriors and lock rod, as well as their metal locking slots. Furthermore, their design was such that they were naturally locked, enabling the box to remain locked in the absence of power (advantageous under situations where packages are placed inside but power has run out). Unfortunately, this same advantage was also a significant disadvantage: the only way to keep it unlocked was by constantly supplying power to it. This was highly suboptimal because a significant portion of our device usage was waiting with an unlocked lid for packages to be placed. Thus, using a solenoid would apply a significant drain our power resources during this time and substantially decrease our device’s power lifespan.

Therefore, we instead opted to pursue a re-design of our original servo motor locking mechanism. After careful consideration and discussion with our professor, we decided on a design with a single line shaped lock arm. Furthermore, to account for potential weak points of contact, we would encapsulate both the motor and the arm in a special housing to strengthen their resistance. To do this, we designed a form fitting capsule for the servo motor, which we could laser cut and stack to cover its width, before drilling screws through the wood to hold the motor in place. Such a casing would enable us to passively apply counter-pressure on every aspect of the the motor and its arm when miscreants tried to force open the box. The first figure below illustrates a single laser cut capsule, along with the complementary laser cut slot to  hold the motor arm. The second showcases the stacked capsule.

Our laser cut housing designs for the servo motor and lock arm. The left component is for the motor, with a small gap cut for the rotation of the motor arm. The right enclosing is the slot the lock arm rotates into to lock the device.

Our servo motor, left, and slot, right, housings glued together with wood glue.

Though we were able to build these housing, it was fairly challenging. None of use came from mechanical engineering backgrounds and so understanding how to create an effective encapsulation that could passively resist prying was difficult. We remember having several discussions with our professor regarding the best way to approach this before finally understanding how to design a basic solution for the problem. Moreover, securing the motor in its housing presented several additional difficulties. Our initial plan was to drill screws through four slots in the servo motor to secure it to the housing. However, we quickly realized that due to its configuration (see picture below), we could only secure it on one side, leaving the other side free. Moreover, we found inserting the screws accurately through its slots very challenging as it was difficult to measure where to insert them from the housing and further ensure that our cuts were vertical. Despite our best efforts, we accidentally broke through one of the servo motor’s slots, leaving it only secured on one of four points. Confronted with this setback, we changed our plan to filling the gap between the motor and its housing in an effort to secure it at a second point. Our final solution was fitting two wood pieces between the motor’s top and the housing, which provided a very snug and stable accommodation. The image below showcases our solution, with the fully housing attached to the box.

Our full lock mechanism attached to the side of the box. Notice the aluminum lock arm, the screws in the back of the motor, and wooden pieces between the housing and the motor.

Similar to the servo motor’s housing, we also laser cut and glued a specialized slot to hold the lock arm which would be placed on the box’s lid. While this task was simpler than securing the motor, it still presented a challenge in that we had to accurately measure a suitable location to glue it to the box so that the arm reached it just right: far enough from the tip of the lock arm to avoid hitting it, but close enough to ensure enough points of contact are established in case of break-in attempts. We had substantial difficulties measuring where to place the slot because we could only accurately determine a suitable location when the box was almost closed, which was inherently difficult as it meant everything was very dark. In fact, our initial slot placement, which we presented during the final critique was too far back, making the lock extremely susceptible to tugging on the lid. Thus, in the week after the final critique we laser cut new housings to extend the lock slot further to provide a much more secure lock arm fit. The image below illustrates our final solution.

The final locking slot for the lock arm.

Lastly, we opted for what we thought was a sturdy piece of aluminum as the lock arm. While this turned out not to be the case as it could still be bent slightly from pressure, we found that adding the housing points of contact greatly diminished this problem.

The servo lock encased in laser printed wood featuring the aluminum lock.

(Un)locking Method

In our prototype build, we utilized a password checking system to lock and unlock the box. While in theory this approach created a heightened sense of security as the likelihood of someone guessing the correct password would be very low, it presented a couple problems. Firstly, the keypad used in the prototype was unreliable – it could not detect certain key presses. Moreover, while other keypads available to us may have been more reliable, they were necessarily weather resistant, making them not appropriate to use in our project. Secondly, keypads caused the locking to be more complicated than needed. For instance, the same objective could equivalently be accomplished with a key switch, which relied on using a correct key rather than a correct password. For these reasons, we instead opted to use a sturdy key switch locking mechanism, where Emily only needed a single correct key to operate the lock and interact with the anti-theft device. Moreover, our key switch was both weather resistant and occupied a much smaller footprint on the box than a keypad would have. We placed this switch in the top right of the box in an effort to make it easy for Emily, who is right handed.

Turning the key in order to change to calibration mode or package detection mode.

Device Powering

One strong request from Emily during our prototype critique was to solar power our device instead of battery powering it (how it was powered in its prototype design). While this was a very interesting idea as it would enable us to use completely renewable energy at a relatively low cost, it was ultimately unclear how best to integrate it with her property. For instance, we collectively believed the best location to place the panel was on the top of her nearby shed. However, the solar panel’s location with respect to her house meant that it was predominately in the shade, and thus would not provide significant power.

Thus, we decided to use battery power, which was non-renewable, but much simpler to use and integrate. However, we noticed that our batteries would only last a few days on a full charge. This stemmed from the fact that our device required 40 mAh on its default state, and the batteries contained just 2800 mAh. As this discovery made battery power in its current state incredibly impractical, we met the professor to determine potential solutions. One of the first thoughts was to use stronger batteries which could hold up to 15000 mAh. However, we ultimately found these to also be impractical as they cost $100 for four (the number we needed), and only prolonged power lifespan by two weeks. Therefore, we instead proposed to set the Arduino to be powered down most of the time, thereby minimizing energy consumption, and only waking up every now and then to check whether packages exist. Interestingly, however, we found even this solution was impractical as every power down code library we tried could only reduce the energy consumption to between 25 – 30 mAh, which only extended battery life to a about five days on normal AA alkaline batteries. This was far from the battery life we hoped to achieve, which was on the order of months. Unfortunately, we have not been able to find a suitable solution to this problem (our professor wanted to discuss solutions with Emily), and thus we leave future exploration in this task to future work.

Our battery housing freshly glued to the bottom of our device. Notice the handle protruding from the front panel, allowing the users to easily open and replace dead batteries. The cardboard cylinder was used as a makeshift clasp to make sure the glue stuck between the housing and box.

Our professor and us using an oscilloscope to measure the voltage difference between a shunt resistor. This helped us determine the default state energy consumption of our device with and without powering down.

Using the sliding drawer to replace batteries.

Other hardware components

In addition to the components discussed in the above sections, we also implemented a pressure sensor reset button. In our discussions with our professor and TA, we recognized that over time the pressure sensors could deviate from their initial readings as the sensors deteriorate. Thus, we added a simple button which allows the device’s own to tare the pressure sensors on demand and recalibrate their sensing. This ensures the device can accurately determine when packages have been placed. We note, that users can only tar the device once its been unlocked (indicating the owner is present) to ensure no unwanted resetting is performed. In addition, we also added a roller arm switch which serves to detect whether the box lid is closed or open. With this, the device is able to recognize when the lid is closed, and only then locks the box if there exists a package. This prevents the dangerous scenario where the motor arm locks while the lid is still open, which prevents it from closing properly and thus becomes very susceptible to package theft.

Our pressure sensor recalibration button, and roller arm switch components in the interior of the box.

Pressing the sampling button as a taring function

Software Integration

Completing our final project required significant restructuring of our code. In this section we cover each component which lead to substantial changes. Additionally, please refer to the bottom section where we embedded our final code for additional explanation and illustration for what we discuss below.

Our most significant code alteration and implementation detail was implementing the package detection logic. In our prototype, we proposed a solution which simply judged if a package was placed based on the most recent pressure reading. Unfortunately, this made our detection highly susceptible to false positives (the device detects a box when in fact it does not exist) due to occasional erratic pressure readings. This is very suboptimal for our task as the device might lock before a package is present. Thus, we instead formulated a different detection metric based on readings over a certain amount of time. Our motivation behind this was: (i) knowing readings over a past timespan can give us a better understanding of whether or not a package exists, and (ii) by conditioning on past history we substantially decrease the likelihood of false positives. The latter we believed to be true because abnormal pressure readings occurred infrequently in our experiments, making it very unlikely that these irregularities would dominate our recorded history. Therefore, we chose to record the ten most recent pressure readings, and continuously update them through each code cycle. Then, determining whether a package had been placed simply entailed checking to see that enough readings from our past history were above a certain package detection threshold. We note that though this detection metric based on history may seem relatively elementary, it worked remarkably well for our use case.

The second most impactful code implementation was our pressure sensor recalibration. As discussed in the previous section, a notable drawback from our pressure sensors was that over time their readings may stray from their original ones. This meant that the sensor could read different values under the same conditions, based on how much use it had seen. This observation immediately raised the concern of heightened false positives or false negatives (a package is placed but the device does not detect it and remains unlocked) as time went on. Indeed, the readings could increase or decrease allowing each of these to occur. For instance, the default readings could increase past the threshold and thus generate false positives, or conversely they could decrease significantly below the threshold such that even with a package on they lie below it and thus create false negatives: the aggregate reading would lie below the detection threshold and the device would remain unlocked. In our attempt to solve this issue, we proposed adding a push button, which allows the user to dynamically change the package detection threshold so that it is inline with the default readings of the sensors, and therefore minimize the likelihood of false negatives or positives. Our solution leverages our pressure reading history when calculating new thresholds, and further adds a small epsilon to account for abnormal pressure readings.

Our third change involved restructuring the device state settings. Initially we proposed to use a keypad password check to distinguish between Emily and miscreants but after discussing with her and our professor we realized that solutions could be significantly easier. For instance, we could replace the keypad – which was not weather resistant or reliable – with a key switch that was both reliable and weather resistant This was our proposed solution, and the code was very straightforward as it was simply switch logic. Similarly, our roller arm switch discussed in the previous section was very simple to implement as it was also just switch logic. The code embedded below exhibits our integration of each.

Conclusion and Lessons Learned

Final Critique Feedback

One of the biggest issues raised from our written feedback and discussions during the final critique was that our device cannot handle multiple package deliveries. For instance the question ‘what about multiple packages in a day?’ was asked more than once. While this is a true and important issue, we intentionally designed our anti-theft device to only handle single time package deliveries chiefly because Emily only expects a single delivery on any given day. Thus, as our primary goal is to create a product that is specific to Emily’s needs, we focused on single delivery anti-theft device solutions. However, scaling our approach to multi-delivery days would be an interesting task. The immediate challenge would be designing a lock which could discriminate between a package handler and Emily, and a miscreant, so that it only unlocks when package handlers or Emily are around. For such cases, it may be useful to utilize a passcode check as the discriminator.

Another important point regarded our lid closing system. Specifically, there was ‘concern [the] lid could slam down’ on the device top and damage it. This was a valid concern, and one which we had thought about in our initial product design, but ultimately did not have enough time to incorporate. However, we recognize that hardware which could solve this issue exists in our lab. Thus this would be very viable point to attend to in future work, and furthermore potentially improve the longevity of our device as it would become less susceptible to slamming damage.

A third critique discussed was that the current ‘lock might need to be stronger’ because ‘a thief could easily break open the box’. While this is partially true as although we apply substantial points of contact along the entire locking arm and motor to counteract malicious prying, it can only withstand a certain amount of pressure. We do note however, that our system satisfied the standards presented by Emily the crimes we collaboratively focused on solving were those of opportunity. In these cases, it is unlikely that potential thieves would dedicate a significant amount of time to prying open locked objects. Nonetheless, there is definitely room to improve  our locks to resist more dedicated thieves. One possible solution might be to convert our servo motor to a linear actuator by transforming its rotational motion to linear motion. This would enable us to emulate the strengths door lock solenoids, which are robust to break-in attempts.

Working with Emily was such a privilege. She was extremely knowledgeable in regards to a lot of the fabrication process as well as the human-machine interaction. Emily eagerly worked with us and had a clear understanding of the difficulties of different aspects of the project. She was also very transparent the entire time, she was not afraid to tell us what she wanted and if she thought we were headed the right direction. During our prototype critique meeting, she gave a lot of constructive feedback that was extremely helpful for developing the final model. Emily’s background made her a very easy partner to work with and we all really enjoyed doing this for her. We hope the project serves her needs well.

This project served as great tool for learning the complexities of tackling real life and impactful projects. Throughout our creation and design process we faced many challenges and setbacks, which illustrated the unexpected difficulties of different processes. Looking back, we dedicated a significant amount of time to creating our solution and are satisfied with the final result. However, there are many things we would have done differently had we know about possible related difficulties in the future. For instance, had we known about the fragility about the pressure sensors, we might have opted to either purchase better quality and more expensive ones, or change our sensing direction to break beam sensing (which we had greater success with in previous projects). Secondly, had we known about the extremely impractical energy consumption of our device, we would have opted for a more energy efficient Arduino chip over the Arduino uno. Overall however, we do not regret the mistakes we made or challenges faced throughout the project as we would not learned as much about the different parts required to put together a fully functioning product.

Technical Details

Code

/*
 * Title: Package Thief Prevention Box
 * 
 * Code Function Overview:
 * This code provides the functionality for our anti-theft prevention box. The comments below 
 * illustrate the function and use of each code element, but a summary explanations is the 
 * following. At the highest level, our code keeps track of the past ten pressure readings
 * our pressure sensor (fsr) recorded. This history is infinitly updated through each 
 * iteration loop cycle. With this frame, the device lies in one of two states, determined by
 * the orientation of the key switch pin. 
 * 
 * When the switch is 'locked' (HIGH), the box is in 'package detection mode' and greadily 
 * searches for potential packages. Once a potential package has been detected, the device 
 * checks its history to determine whether the reading was a fluke or an actual package and 
 * locks accordingly. Note however that the device will only lock if it has not already been 
 * locked, a package exists, and the lid is closed. Conversely, if a package is not detected, 
 * then the box stays unlocked. 
 * 
 * When the key switch is 'unlocked' (LOW), the device is in 'package removal and configuration
 * mode'. In this state, the device unlocks (if it is currently locked), which enables the user 
 * to lift packages out of it. Furthermore, this mode also allows the user to recalibrate the 
 * sensor via taring, by pressing the sample pin. 
 * 
 * Inputs:
 *  - Pressure sensor 
 *  - Key switch
 *  - Sample button for taer recalibration
 *  - Roller switch for lid closing detection
 * Outputs:
 *  - Motor arm locking or unlocking
 * 
 * Credit:
 * We use the Adafruit SleepyDog package to power down our arduino for power savings. The
 * code can be reached following this link: https://github.com/adafruit/Adafruit_SleepyDog
 * 
 */

#include <Adafruit_SleepyDog.h>
#include <Servo.h>

// Record pressure pin
const int PRESSURE_PIN = A1;
// Record Servo Pin
const int SERVO_PIN = 10;
// Record Key Switch Pin
const int KEY_PIN = 9;
// Record Roller Switch Pin
const int ROLLER_PIN = 8;
// Record Sample Button Pin
const int SAMPLE_PIN = 7;
// Initialize MOTOR
Servo MOTOR;
// Initialize device state variables
// Record whether a package is place in the bin
bool PACKAGE_EXISTS = false;
// Record if the lid is closed
bool LID_IS_CLOSED = false;
// Record if the bin is locked
bool BIN_IS_LOCKED = false;
// Record the state of the key switch. 
// Note: 
//  - HIGH: Key in the 'lock' position
//  - LOW: Key in the 'unlock' position
int KEY_LOCKED = HIGH;
// Record whether to take a sample of pressure sensors.
// This is used to set a new threshold for package detection.
//  - HIGH: Do not take sample
//  - LOW: Take sample
int TAKE_SAMPLE = HIGH;

// Instantiate initial thresholds
// Unsigned to: (i) Increase INT_MAX, (ii) pressure should never be < 0
unsigned int PRESSURE_THRESHOLD = 0;
// Amount of history to record
unsigned int HISTORY_AMOUNT = 10;
// Initialize pressure history sequence
unsigned int PRESSURE_AMOUNTS[10] = {0, 0, 0, 0, 0, 0, 0, 0, 0, 0};
// Record current iteration in the device state. Also is used to update pressure amounts list
unsigned int ITERATION_NUM = 0;
// Hyper parameter indicating minimum proportion of times the pressure reading is above the 
// pressure threshold for a package to be deemed to exist.
float PROPORTION_THRESHOLD = .7;

void setup() {
  // Dictate Modes of all buttons, MOTORs, and switches
  MOTOR.attach(SERVO_PIN);
  pinMode(ROLLER_PIN, INPUT_PULLUP);
  pinMode(KEY_PIN, INPUT_PULLUP);
  pinMode(SAMPLE_PIN, INPUT_PULLUP);
  MOTOR.write(110);
  // Explicitly write arduino LED pin to LOW to conserve energy
  pinMode(13, OUTPUT);
  digitalWrite(13, LOW);
}

void loop() {
  // Power arduino down for half a second to minimize energy consumption.
  Watchdog.sleep(500);

  // If the iteration is larger than the history amount, roll it over to the beginning.
  // This allows us to continuously update the pressure history.
  if (ITERATION_NUM >= HISTORY_AMOUNT) {
    ITERATION_NUM = 0;
  }

   // Read the current pressure
  unsigned int pressure_read = analogRead(PRESSURE_PIN);
  // Update pressure history with current pressure reading
  PRESSURE_AMOUNTS[ITERATION_NUM] = pressure_read;
  // Read if lid is closed, lock is in closed position, and whether to take a sample
  LID_IS_CLOSED = digitalRead(ROLLER_PIN);
  KEY_LOCKED = digitalRead(KEY_PIN);
  TAKE_SAMPLE = digitalRead(SAMPLE_PIN);
  // Key is in the unlocked position
  if (KEY_LOCKED == LOW) {
    // If the box is locked, unlock it.
    if (BIN_IS_LOCKED)  {
      // Indicate that the box is now unlocked. This ensures that we do not redundantly 
      // make the MOTOR unlock an already unlocked box.       
      BIN_IS_LOCKED = false;
      // Unlock box
      MOTOR.write(100);
      // Delay to give MOTOR time to unlock the box
      delay(1000);
    }
    // Take sample if desired. Assumes that the package is already outside the box.
    // Note that a sample can only be taken if: (i) key is in unlocked position 
    // (box is unlocked), (ii) the box lid is open (gives us an indication that
    // that the person has removed the package, (iii) person requests a sample to be taken.
    if ((LID_IS_CLOSED == HIGH) && (TAKE_SAMPLE == LOW)) {
      // Reset pressure threshold
      PRESSURE_THRESHOLD = 0;
      // Compute new pressure threshold. We take the max of all pressure history to
      // Try and account for 'worst case' abnormal pressure readings.
      for (int i = 0; i < HISTORY_AMOUNT; i++) {
        PRESSURE_THRESHOLD = max(PRESSURE_THRESHOLD, PRESSURE_AMOUNTS[i]);
      }
      // Add a small epsilon of error to account for small variations in pressure
      // readings in the packageless state.
      PRESSURE_THRESHOLD+=5;
    }
    
  }
  // Box is in package detection mode
  else {
    // Lid is not locked and a potential package has been sensed.
    if (PRESSURE_AMOUNTS[ITERATION_NUM] > PRESSURE_THRESHOLD && !BIN_IS_LOCKED) {
      // Look at past pressure reading history to determine whether a potential package exists.
      float package_readings = 0;
      for (int i = 0; i < 10; i++) {
        int pressure_reading = PRESSURE_AMOUNTS[i];
        // Current reading is above our threshold for determining whether potential packages exist.
        if (pressure_reading > PRESSURE_THRESHOLD) {
          // Record observation
          package_readings++;
        }
      }
      // Compute the average number of times a potential package has been detected 
      // over the past history.
      package_readings = package_readings / HISTORY_AMOUNT;
      // A potential package has been detected enough times to determine it is 
      // an actual package.
      if (package_readings > PROPORTION_THRESHOLD) {
        PACKAGE_EXISTS = true;
      }
      // A potential package has not been detected enough times to determine it
      // is an actual package.
      else {
        PACKAGE_EXISTS = false;
      }
    }
    // No potential package has been detected
    else {
      PACKAGE_EXISTS = false;
    }
    // Lock box if a package exists, the lid is closed, and the box is not already locked.
    // Note that by tracking if a box is already locked, we eliminate redundant commands to 
    // to the MOTOR regarding locking the box.
    if (PACKAGE_EXISTS && (LID_IS_CLOSED == LOW) && !BIN_IS_LOCKED) {
      // Lock the box
      MOTOR.write(35);
      // Inidicate that the box is now locked
      BIN_IS_LOCKED = true;
      // Delay to give MOTOR enough time to lock the box
      delay(1000);
    }
  }
  // Increase iteration number to next step in history
  ITERATION_NUM++;
  

}

Schematic and Design Files

Team Emily CNC and Laser Cutting

Previous Posts

Meeting with Kathy

Prototype Documentation

What we built

The Light Switch 2.0 allows the user to time operation of the light switch such that it automatically turns off after a 10-minute delay, but also remains unobtrusive enough such that the user can still operate the light switch in an ordinary manner.

Overall photo showing the project.

The switch mechanism does not obstruct usage of the switch, but some of it is visible.

The control panel which houses the buttons and LED. The left button operates the switch instantly; the right button starts a timer process.

The panel lifts to expose the batteries for easy access.

When the timer button is pressed, the LED lights up to indicate as such.

Close up of protoboard wiring.

Close up of rack and pinion.

Looking at the battery panel. It uses a pack of four AA batteries and should run for a long, long time.

The switch is fully operable as a normal light switch.

How this project would look mounted on a wall.

Narrative Sketch (How would Kathy use this?)

Kathy has just had a wonderful dinner with her guests and everyone’s had their fun but it’s getting late. She needs to see them out to her doorstep where guests can bid their farewells and head out to their cars, so she activates the Light Switch 2.0 attached to her porch light. Then, she returns upstairs to where she can watch them leave from her living room window. Because the light will automatically turn off in 10 minutes, she has no need to go back downstairs and turn the light off again. This saves her another trip down the stairs.

How we got here

One of our biggest hurdles was that meeting with Kathy was initially unfruitful; we struggled to grasp at a proper project idea. A series of further meetings confirmed a concrete idea, however.

Brunch at Kathy’s with the crew!

Our humble beginnings started with some very basic materials, starting from a plastic straw stuck on a servo.

Servo testing. We tested for how strong the grip on the servos would have to be and how high.

We also cobbled together a bunch of Popsicle sticks for a stand as a sort of functional prototype holder.

The progression of the temporary servo-holders.

Our work resulted in a simple, behaves-like functional prototype. It turned a switch on and off, but using simple and homely means, as well as with a few buttons, a rotary encoder, and a LCD screen.

Where we stood with the prototype.

An early sketch of the new and improved project.

Predictably, the critique gave us valuable feedback to modify our project. Of course, we knew that our use of servos would have to be rethought. Originally our plans would have been to use a linear actuator but based on time and cost,we used gears and a servo to mock that functionality.

Based on prototype critiques made by Kathy, we ended up having to rethink the entire user interface of our project. We decided to limit the functionality of our item given that it was all we needed and frivolousness would have been in poor visual taste. No longer would we have so many moving parts; instead just a 10 minute hard-coded timer would replace the LED screen and rotary encoder.

Starting over from scratch. All the plans have completely changed, and we’re using a larger servo now.

A simpler process would replace the circular motion of our dual servos from the initial prototype. Using gears and a single powerful servo, we mimic a linear actuator to more reliably switch it on and off.

Final mechanism used to push the switch. Laser-cut out of acrylic.

We made a holder made out of metal, mostly courtesy of Justin’s CNC routing skills. It is to be the base for the system and the cover.

Metalworking for the cover via CNC routing.

The finished foundation. The servo runs the gear, which pushes the linear piece through the groove.

Despite some soldering problems, moving onto a more convenient proto-board helped wrap up the software side of things and enabled us to put all the pieces together. We used a set of four batteries to enable the user the most battery life possible, as it should last for a long time. This was at the expense of the visual appeal, as it added a bit more bulk to the overall product.

The piece with all the hardware in place.

We went through the critique coverless, but that was an issue we still wanted to address. For the cover we decided upon 3D printed plastic. We encountered a problem with button sizes, but ended up sanding the excess away. In addition, the panel was slightly short for the contents, but slightly. We decided upon hot glue to coax the seam a little more together.

The 3 pieces of the cover, screwed on.

Conclusions

Feedback from the Public

“Battery could be made smaller with a Lipo battery”

“Why batteries and not get it from house?”

Some concerns with the batteries arose during the critique. We had a few reasons for batteries over wall-switch power, mainly to do with the layout of the house, but it would be wise to consider batteries of smaller size. We wanted to maximize the power usage rather than minimize size, but if the size is that much of an issue, we would consider this for future iterations.

“Somewhat large/clunky box on light switch”

“Form function, Electronic vs mechanical.”

“more room for switch access.”

Many of the critiques centered around how large the box is attached to the switch. The bulk of the size is the battery and servo, and had we been able to locate smaller parts with the same or better functionality, we would have preferred to use those. If there was a method to better solve this problem with smaller parts or even at all smaller parts, we would gladly have it.

However, one piece of salient advice that we would consider in later iterations is the switch access, which could be perhaps solved with a different cover with lower walls.

“Not multiple timers.”

This functionality was voted off by our person, who wanted minimalism and functionality rather than more UI. We originally had an adjustable timer, but our client did not need it. Nevertheless, we agree that perhaps to make this overall a more useful product a way to change the timer would be helpful.

“Works, simple to use.”

“Nice machining. The lever to turn the switch on and off is sold and seems to be unaffected by electrical concerns.”

Much of the praise for the product centered around the mechanical and functional simplicity of our product. It comes at odds with the comment critiquing the simplicity of the product. I think we have settled in a good sweet spot for Kathy, but not necessarily for everyone, and perhaps in future iterations a small amount of tweaking could fix that.

“Sort of roundabout way of solving the problem, better solved by smart lighting.”

The discussion we had with Kathy regarding this problem could have been better answered by an electrician, but as we were not electricians, we could only help with an external mechanic. However, we agree: perhaps some wall-rewiring would also solve this problem.

Major takeaways

It was a really pleasant experience to work with Kathy on this. She made great effort to be helpful throughout our process, and we were able to discuss quite a lot. We really didn’t have any complaints about working with her. The only thing that may have smoothed the process over might have been a more set agenda when meeting with her, as we may have discovered other problems to fix earlier.

Regarding the build process, there were a few issues that could have been better addressed. First, the bottle-necking that parts-sourcing created gave us a schedule shift in a negative direction. We were preoccupied with finding a linear actuator to buy and ended up not using any. Had we decided on a servo automatically, the process would have been much more streamlined. In addition, some parts of the process take more time. and should have been considered. The 3D printed cover took a while to print, and therefore should have been allotted more time on the schedule, and having to create the servo mechanism put a lot of pressure on Justin. The group roles could also have been better assigned to maximize efficiency. Mechanical engineering provided the bulk of time-consuming process, and could have been a burden better shared.

Technical details

Code

/* This code controls a servo motor to flip a light switch
 *  after 10 minutes or on demand depending on which button
 *  is pressed.
 *  Components:
 *  1 x 270 degree servo motor
 *  2 x tactile momentary push buttons
 *  1 x LED
 *  Inputs: button presses
 *  Outputs: blinking LED, servo motor movement
 */
const int START = 2;
const int MANUAL = 4;
const int SERVO = 3;
const int LED = 1;

unsigned long timer = 0;
bool state = false;
bool start = false;

void setup() {
  pinMode(SERVO, OUTPUT);
  pinMode(MANUAL, INPUT_PULLUP);
  pinMode(START, INPUT_PULLUP);
  pinMode(LED, OUTPUT);
}

void moveUp() {//turn light switch on
  int i = 0;
  while (i < 100) {
    digitalWrite(SERVO, HIGH);
    delayMicroseconds(2000);
    digitalWrite(SERVO, LOW);
    delay(10);
    i++;
  }
  resetServo();
}

void resetServo() {//reset to middle resting position
  int i = 0;
  while (i < 100) {
    digitalWrite(SERVO, HIGH);
    delayMicroseconds(1200);
    digitalWrite(SERVO, LOW);
    delay(10);
    i++;
  }
}

void moveDown() {//turn light switch off
  int i = 0;
  while (i < 100) {
    digitalWrite(SERVO, HIGH);
    delayMicroseconds(600);
    digitalWrite(SERVO, LOW);
    delay(10);
    i++;
  }
  resetServo();
}

void blinkLED() { //function that blinks an LED 4 times
                 // to indicate that the timer has started
  digitalWrite(LED, HIGH);
  delay(100);
  digitalWrite(LED, LOW);
  delay(100);
  digitalWrite(LED, HIGH);
  delay(100);
  digitalWrite(LED, LOW);
  delay(100);
  digitalWrite(LED, HIGH);
  delay(100);
  digitalWrite(LED, LOW);
  delay(100);
  digitalWrite(LED, HIGH);
  delay(100);
  digitalWrite(LED, LOW);
}


void loop() {
  //state = true when light is in ON position and false
 // for OFF position
  if (!digitalRead(MANUAL)) {
  /* Check to see if the manual operation button has been pushed.
   * If it has, turn the light on or off depending on previous state.
   */
    if (state) {
      moveDown();
      state = false;
    }
    else {
      moveUp();
      state = true;
    }
    /* If manual button was pressed before the 10 minutes have passed
     * after pressing the start button, the timer is overrided and manual
     * light switch operation is enabled.
     */
    start = false;
  }
  if (!digitalRead(START)) {
  /* When the start button is pressed blink the LED and begin
   * the timer
   */
    blinkLED();
    start = true;
    timer = millis();
  }
  if (start) {
    if (millis() - timer >= 600000) { //check to see if 10 minutes have passed
      moveDown(); //if 10 minutes have passed, turn light switch off
      timer = millis(); //reset timer
      state = false;
      start = false;
    }
  }
}

Schematic and design files

DXF Files Here

What We Built

We created a device that enables users to perform several different physical exercises such as bicep curls, lateral raises, and many other resistance band exercises in a home gym setting. They are then able to access information on their performance including max weight, the number of repetitions, and how their left and right sides compare to each other. To accomplish this, we decided on anchoring a sensor connected to a resistance band to the base of a door since we saw this as a convenient location that everyone would have in their homes. Adjacent to the sensor is an interface that users can use to see the results of their exercises.

The device has a main display that welcomes the user and then prompts them to begin an exercise on either their left or right side. Once a side is selected, there is a brief countdown to allow the user to get into position and begin the exercise. After the exercise is completed, the user can either perform another exercise on the other side or view their results at any time.

There is an adjustable metal bracket that allows the device to be anchored underneath doors of varying widths. Foam/rubber pads protect the door and prevent the device from sliding horizontally along the bottom of the door.

Connected directly to the metal bracket is a load cell that measures the force pulling on it from a resistance band. The load cell is connected in such a way that it can pivot in a 90 degree arc allowing for many more exercises to be done.

On the bottom side of the device is a removable panel that gives users access to the battery holder in order to change batteries once they are worn out. Additionally, to save battery, there is an on/off switch on the side of the device so that the Arduino only draws power when the device is in use.

The device is anchored by ensuring that the adjustable bracket is set to the correct width and then sliding it underneath a door.

A user performing a bicep curl.

Users can change the resistance by stepping further/closer to the door and then performing their exercise.

By pressing buttons on the device, users can change sides, switch exercises, and view the results of their workout.

The following is a brief narrative sketch to help describe how our device fits in the context of everyday life:

Gary suffered a stroke and has been working hard in his home gym to correct the resulting muscle imbalance. While Gary has a trainer to help him do this, neither Gary nor his trainer have a way to quantitatively measure fine differences in strength on either side of Gary’s body.

At the beginning of each week, Gary performs several exercises such as bicep curls, lateral raises, and leg lifts using the device, and notes which side of his body is stronger and by how much. He is then able to work with his trainer to structure his week’s workout routine with the goal of balancing his strength accordingly. He is able to use the device at any point (similar to how one might use a scale to check their weight) in order to see his progress and have hard numbers to work with in improving his physical strength and becoming more balanced.

How We Got Here

The load cell sensor we were worked with

One of the first decisions we made as a team was the choice of sensors. Due to the form factor and flexibility we needed in our solution, a load cell seemed like the best bet, but we also initially considered using pressure sensors. The choice of load cell was the decision we look back on and wish we could have changed. We selected a 40kg bearing load cell from Amazon, which also came with an integrated board that amplified signals from the sensor and worked perfectly with an Arduino. Not only was the hardware supposed to integrate perfectly with our Arduino, but the product included a well written tutorial on how to capture signals from the load cell. In being content with the tight integration with our Arduino, however, we failed to consider just how flimsy and difficult the load cell would be to work with.

Resoldering the broken wires on the load cell

The load cell was designed to support 40kg, but the wires that detected changes in force were extremely thin and connected to the sensor with a fine plastic coating. This coating barely held the wires in place and we ended up tearing one of the wires within a few days of development. Attempts to solder it back were fruitless because there was limited wire to metal contact and the plastic coating had burned. In the end, a new, second sensor that we took extreme precaution with also broke, leading us to conclude that a more durable load cell would have been better from the get go. In fact, we have ordered a new one to be used in the final product we give to Gary.

Full electronic prototype, with working load cell, buttons, and screen interface

The lesson we learned from this was to first analyze how a physical component would perform in the real world as opposed to determining its simplicity for use with hardware or software. The sensor not only created trouble when assembling and testing our device, but also in the software development phase because we didn’t know if the problem was with the physical load cell or with the software. Before we knew that it was the weak wiring that was causing the problem, this was highly frustrating.

One of the challenging aspects of our project was repetition counting and weight measurement. After discussing with our professor and understanding the processing limits of the Arduino, we decided to implement a windowing approach. Our code kept a few hundred milliseconds of data and averages in buffers and calculated local maxima based on the slope of the data.

Signals captured from exercise repetitions, when the sensor was working

When we tested this on actual measurements from the device, it worked extremely well. This was a breakthrough moment for us and helped refocus our efforts on the physical and reliability aspects of the device. In general, the software aspect of our project was not very time consuming and we were done with that part ahead of schedule.

Testing the load cell while it was not working, not knowing what the problem was (feat. helpful non-class member)

Frustration with device – it was not communicating with the computer

The physical construction of the device was something we discussed at length. At first, we were debating whether to use a device mounted to the door (as in our final product) or to use something in line with a resistance band, to measure the force exerted by pulling on both sides of the band. Then, we needed to decide how best to house the load cell sensor as well as the screen and buttons interface – we considered having two parts to the project, so that the screen and buttons could perhaps be a separate handheld component. However, we decided against this as it would require two Arduinos and thus two sources of power, making the whole setup less clean. We eventually settled on a door-mounted, single-box design.

Sketches of ideas for housing the device and load cell

Assembling the housing using plastic solvent

Fitting the electronics into the housing. Finding an intuitive way to organize the Arduino, sensors, battery holder, etc. proved more challenging than we initially thought.

We didn’t follow our schedule too well and underestimated how long it would take to physically wire up and build the device, but were able to manage the rest of our time fairly effectively. This was possible because we delegated certain aspects of the project to each group member according to their strengths.

Conclusions and Lessons Learned

Several people stated that they “would like to see a smaller device” more in line with the resistance band, as something smaller might be more convenient to use on a regular basis. We had already recognized this as a way to continue improving our design, and were relatively unsurprised that the size of the device was brought up by multiple people. However, a comment stated that it was unfortunate that the user had to bend down to press the buttons, which is a good point. We thought about ways we could have fixed this, but couldn’t find any solutions that didn’t involve having another, separate part of the device,

Many of our reviews mention that the device “feels fragile and makes the user afraid to apply full force”, although the force is not applied directly to the box. During the final crit, we found ourselves explaining this to people repeatedly, which is a sign that we should have made our device visibly appear more sturdy. We believe this perception of our device could be due to the transparency of the acrylic used for the housing, although this has no impact on its actual strength. We did not realize how much this would affect people’s perception of the device.

Our device intrigued the Osher students who observed it, and throughout the crit we were asked if the device could be applied to multiple different exercises. We showed them different ways to hold the resistance band to work different muscles, but a prevailing idea seemed to be to provide multiple attachments for the device (not just the resistance band). Our device is already capable of doing this, since the attachment would just need to be able to connect to a carabiner, but this feedback reinforced the idea of preserving the adaptability of our device to multiple exercises and situations. For example, having our device be able to handle “different door placements”, different exercises, and different amounts of force.

Working with Gary was a pleasure – he had a clear idea of what he wanted from the very beginning, which surprised us but was very helpful in getting us started. He was always available for feedback, and seemed very happy with the direction we were going in. However, we were working somewhat blindly, as he didn’t tell us what specific exercises he did and we just worked off our best guess of what he needed. We happened to make the right choices, but were I to do this again I would make sure to ask for all the information we need from the start.

Overall, we had a relatively even distribution of work over time, but fell off schedule and spent a large portion of a day putting everything together. Doing this again, we would attempt to stay on track and finish different parts of the project by the scheduled times. This way, we would be able to achieve our longer-term goals, such as making our design more compact by using a smaller Arduino or microcontroller.

It was a very rewarding experience to be able to make something suited to someone we have been talking to for over a month, and have it suit his specific needs, but also be applicable to many other situations. Gary actually wants to take this project forward and develop it into a more marketable product, which is unexpected, but it is very exciting that Gary thinks that our product has potential in the market.

Technical Details

Code

/*
 * Strength Balance
 * 
 * Description: We created a device that enables users to perform several different physical exercises 
 * such as bicep curls, lateral raises, and many other resistance band exercises in a home gym setting. 
 * They are then able to access information on their performance including max weight, the number of 
 * repetitions, and how their left and right sides compare to each other. To accomplish this, 
 * we decided on anchoring a sensor connected to a resistance band to the base of a door since we saw 
 * this as a convenient location that everyone would have in their homes. Adjacent to the sensor is an 
 * interface that users can use to see the results of their exercises.
 * 
 * 
 * This code takes load cell data, processes it, and displays the results on a  
 * screen. It also responds to user input, by means of thee buttons.
 * Input: 3 buttons, 1 load cell sensor || Output: 1 LCD screen
 * 
 * PIN 2  = Load Cell Sensor Pin CLK
 * PIN 3  = Load Cell Sensor Pin DOUT
 * PIN 8  = Clear Button
 * PIN 9  = Right Button
 * PIN 10 = Left Button
 * 
 */
#include <SPI.h>
#include <Wire.h>
#include <LiquidCrystal_I2C.h>
#include "Queue.h"
#include "HX711.h"

#define DOUT  3 // Load Cell pin 1
#define CLK  2 // Load Cell pin 2
#define BUTX 8 // Clear Button
#define BUTR 9
#define BUTL 10
#define CALIBRATION_FACTOR -7050
#define WINDOW_SIZE 5
#define AVERAGES_SIZE 5

LiquidCrystal_I2C screen(0x27, 20, 4);

HX711 scale;

const char LEFT = 0;
const char RIGHT = 1;

float calibrationFactor = -7050; 
//calibration factor used by HX711 library to do some sort of data filtering
unsigned long startMillis;
unsigned long timerMillis;
unsigned long currentMillis;
int exerciseTimer = 0;
int rightPressed = 1;
int leftPressed = 1;
int clearPressed = 1;
bool exerciseMode = false;
int leftMax = 0;
int rightMax = 0;
bool leftActivated = false;
bool rightActivated = false;
bool leftInUse = false;
bool rightInUse = false;

// window and average queues
Queue<int> window = Queue<int>(WINDOW_SIZE);
int windowSum = 0;
Queue<float> averages = Queue<float>(AVERAGES_SIZE);
float averagesSum = 0;
// variables to help with repetition counting
String oldSlope = "flat";
int repLeftCount = 0;
int repLeftWeightSum = 0;
int repRightCount = 0;
int repRightWeightSum = 0;

//---  HELPERS  ---

void initLoadCell()
{
  scale.begin(DOUT, CLK);
  scale.set_scale(CALIBRATION_FACTOR);
  scale.tare(); //Reset the scale to 0
}

void homeScreen() {
  screen.home();
  screen.clear();
  screen.print("Press L/R to begin");
  screen.setCursor(0, 1);
  screen.print("and end exercises.");
  screen.setCursor(0, 2);
  screen.print("Press X for results.");
  screen.display();
}

void beginExDisp() {
  for (int i = 5; i > 0; i--) {
    screen.home();
    screen.clear();
    screen.print("Start in ");
    screen.print(i);
    screen.print(" seconds");
    screen.display();
    delay(1000);
  }
  screen.clear();
  screen.print("Go!");
  delay(500);
  screen.display();
  startMillis = millis();
  exerciseTimer = 0;
}

void handleButtonSide(char side) {
  beginExDisp();
  if (side == LEFT)
  {
    leftActivated = true;
    leftInUse = true;
    rightInUse = false;
  }
  else
  {
    rightActivated = true;
    leftInUse = false;
    rightInUse = true;
  }
}

void handleButtonClear() {
  // if both the left and right buttons were pressed during 
  // the last exercise set
  screen.clear();
  if (leftActivated && rightActivated) 
  {
    screen.print("Left Max: ");
    screen.print(leftMax);
    screen.setCursor(0, 1);
    screen.print("Reps: ");
    screen.print(repLeftCount);
    screen.setCursor(0, 2);
    screen.print("Avg Weight: ");
    float leftAvg = (float)repLeftWeightSum / (float)repLeftCount;
    screen.print(leftAvg);
    delay(3000);
    screen.clear();
    screen.print("Right Max: ");
    screen.print(rightMax);
    screen.setCursor(0, 1);
    screen.print("Reps: ");
    screen.print(repRightCount);
    screen.setCursor(0, 2);
    screen.print("Avg Weight: ");
    float rightAvg = (float)repRightWeightSum / (float)repRightCount;
    screen.print(rightAvg);
    delay(3000);
    if (!(leftAvg == 0)) {
      screen.clear();
      screen.print("Right : Left = ");
      screen.setCursor(0, 1);
      screen.print(rightAvg);
      screen.print(" : ");
      screen.print(leftAvg);
      delay(3000);
    }
  }
  else if (leftActivated)
  {
    screen.print("Left Max: ");
    screen.print(leftMax);
    screen.setCursor(0, 1);
    screen.print("Reps: ");
    screen.print(repLeftCount);
    screen.setCursor(0, 2);
    screen.print("Avg Weight: ");
    float leftAvg = (float)repLeftWeightSum / (float)repLeftCount;
    Serial.print(leftAvg);
    screen.print(leftAvg);
    delay(3000);
  }
  else if (rightActivated) 
  {
    screen.print("Right Max: ");
    screen.print(rightMax);
    screen.setCursor(0, 1);
    screen.print("Reps: ");
    screen.print(repRightCount);
    screen.setCursor(0, 2);
    screen.print("Avg Weight: ");
    float rightAvg = (float)repRightWeightSum / (float)repRightCount;
    Serial.print(rightAvg);
    screen.print(rightAvg);
    delay(3000);
  }
  else
  {
    screen.print("No reps counted!");
    delay(500);
  }

  // reset globals
  leftMax = 0.0;
  rightMax = 0.0;
  leftActivated = false;
  rightActivated = false;
  leftInUse = false;
  rightInUse = false;
  homeScreen();
  initLoadCell();
  repLeftCount = 0;
  repLeftWeightSum = 0;
  repRightCount = 0;
  repRightWeightSum = 0;
  Serial.println("reset reps");
  
}

//---  MAIN CODE  ---

void setup() {
  Serial.begin(9600);
  pinMode(BUTR, INPUT_PULLUP);
  pinMode(BUTL, INPUT_PULLUP);
  pinMode(BUTX, INPUT_PULLUP);
  
  screen.init();
  screen.backlight();
  screen.print("Welcome, Gary!");
  delay(1000);
  homeScreen();

  scale.begin(DOUT, CLK);
  scale.set_scale(CALIBRATION_FACTOR);
  scale.tare();
  
  startMillis = millis();
  timerMillis = millis();
}


void loop() {
  screen.home();
  scale.set_scale(calibrationFactor);
  
  currentMillis = millis();

  rightPressed = !digitalRead(BUTR);
  leftPressed = !digitalRead(BUTL);
  clearPressed = !digitalRead(BUTX);

  if (clearPressed)
  {
    exerciseMode = false;
    handleButtonClear();
  }
  else if (leftPressed)
  {
    exerciseMode = true;
    handleButtonSide(LEFT);
  }
  else if (rightPressed)
  {
    exerciseMode = true;
    handleButtonSide(RIGHT);
  }
  
  // figure out average from window
  int newReading = abs(scale.get_units());
  window.push(newReading);

  float newAverage;
  if(window.count() >= WINDOW_SIZE)
  {
    int oldReading = window.pop();
    windowSum += newReading;
    windowSum -= oldReading;
    newAverage = float(windowSum) / WINDOW_SIZE;
  }
  else
  {
    windowSum += newReading;
    newAverage = float(windowSum) / window.count();
  }

  // figure out up/down from average
  averages.push(newAverage);

  float oldTotal = averagesSum;
  float newTotal;
  
  if(averages.count() >= AVERAGES_SIZE)
  {
    float oldAverage = averages.pop();
    averagesSum += newAverage;
    averagesSum -= oldAverage;
    newTotal = averagesSum;
  }
  else
  {
    averagesSum += newAverage;
    newTotal = averagesSum;
  }

  // decide if up/down, adjust vars
  // figure out the new slope
  float difference = newTotal - oldTotal;
  String newSlope = "";
  if (difference <= 0.02 && difference >= -0.02)
  {
    newSlope = "flat";
  }
  else if(difference >= 0.02)
  {
    newSlope = "positive";
  }
  else if(difference <= -0.02)
  {
    newSlope = "negative";
  }

  // decide if a repitition was performed
  if (newSlope == "negative" && oldSlope == "positive")
  {
    if (leftInUse)
    {
      repLeftCount++;
      repLeftWeightSum += newReading;
      Serial.println("left side reading: ");
      Serial.print(newReading);
    }
    else if(rightInUse)
    {
      repRightCount++;
      repRightWeightSum += newReading;
      Serial.println("Right side reading: ");
      Serial.print(newReading);
    }
  }
  oldSlope = newSlope;

  // update maximum reading
  if (leftInUse && (leftMax < newReading))
  {
    leftMax = newReading;
  }
  else if(rightInUse && (rightMax < newReading))
  {
    rightMax = newReading;
  }
  if (exerciseMode) {
    if (currentMillis - timerMillis >= 1000) { // keep track of time passed while exercising
      exerciseTimer++;
      screen.clear();
      screen.print("Time: ");
      screen.print(exerciseTimer);
      screen.print(" seconds");
      screen.setCursor(0,1);
      screen.print("Working ");
      if (leftInUse) { screen.print("left "); }
      else { screen.print("right "); }
      screen.print("side.");
      screen.display();
      timerMillis = millis();
    }
  }
}

Schematic

Final Design DXF

Richard is a graphic and jewelry designer who typically works on drawings for commissions every evening on the couch while he and his wife watch the news. You can see the notes from our initial meeting here.

While he does this, he often has to tilt and position his iPad in different ways to get the right angle to draw from. He typically keeps it on a pillow and props it up with his hand, which doesn’t allow for it to be fully stable, and makes drawing harder. He tries to keep track of the time he’s spent working on each commission in order to keep pricing fair and know for future reference how long similar designs take.

We worked to create a prototype for him to review and give us feedback.

Prototype

Our prototype gave Richard a sense of the scale of the final product, so as to demonstrate both the movement of the board and to understand and look critically at the experience of using the device to assist you in drawing on an iPad. The stand attaches to the back of the board and sits on a table, where it can be moved up and down to better suit Richard’s desired angle of drawing.

The front of the board holding the iPad was milled to hold Richard’s 12 inch iPad

Back view showing the positioning joint attached to the back of the holding board.

Finger slot to allow Richard to get his iPad back out of its holder.

Process Photos

Our initial sketch for the arm mechanism featuring a sliding slot joint for vertical movement and a ball & socket joint for planar rotation.

Our initial sketches for the digital interface on the board featuring different buttons for each project, a home button, and start and finish buttons, intended to be built with the LED screens in the Physical Computing room.

Initial sketch of how all the components would look on the board.

An initial circuit diagram.

Final concept for electronics interface, coded but with nothing to show as we do not have the display yet, designed around usage of a TFT LED display, which sets a display through individually coded pixels, allowing us to change font sizes and placements to add on-screen variable labels to buttons, cutting down to only three. A rotary encoder allows Richard to enter the name for each project, and an additional power button allows the entire arduino to turn off when not in use, saving battery power.

Prototype board with space for iPad milled in along with a finger slot to get it out again.

We initially tried to attach the camera mount to the board by inserting a hex nut into the board for the camera mount to screw into.

Closer shot of initial attempt to mount board to the camera equipment w were using as a joint. WE tried to embed and glue a nut into the MDF.

Discussion:

Many of the challenges we faced during this process involved getting the right parts to mount the board on. We knew that finding a combination of parts that allowed easy movement using a joint that wouldn’t collapse under the weight of the board, while positioning the board in a way that it wouldn’t tip over or rest too high above whatever surface it was placed on would be difficult, so we tried to focus on the mechanical aspect of this prototype. It took us a long time to find the right parts to order for the joint, which delayed our finding a good way to attach the joint to the board properly, and multiple different iterations of how Richard would interact with the LCD interface delayed us in getting a functional electronic prototype.

In order to be able to use the LED displays already available in the physical computing room, which display text in clusters of pixels and can only allow text inputs in a constant size, we created a system with 7 buttons, allowing us to have a constantly defined button for each conceivable purpose, including three buttons labelled as three different projects. Initially, we envisioned the working projects as simply being numbered as project 1, 2, and 3, but then decided that, as a record of previous projects would be an important part of the purpose of tracking the time for each project, when a project was completed, the next project should be labelled #4 and so on. Because the buttons for project 1, 2, and 3 were to be labelled with those numbers and placed in order, we reasoned that this would be confusing when, for example, project 2 was completed and the list became 1, 4, and 3. Additionally, past the first few projects, the numbers corresponding to different past projects would no longer be reasonable to remember, so we decided to add a method of naming each project. We looked for Arduino compatible keyboards, but decided that that would be an unnecessarily large component that would not fit into our already designed dimensions of the physical board.  Instead, we added a rotary encoder, programmed to enter text letter by letter. We also decided to purchase a TFT display, and created code that would allow each button to be labelled on the display, allowing us to cut down to 3 buttons.

While during the presentation we didn’t have much time to get questions from the audience, we got helpful feedback from Richard regarding ways to make the form easier for him to use, such as adding a finger slot to the side of where we milled a place for his Apple Pencil to rest, and rounding out the right edge of the board so that his forearm wouldn’t chafe when he was drawing. He experimented with angles at which he would usually hold the iPad, and found that the fit was secure, but asked if we could add elastic on corners or hollow out the space below the top edge for a snugger fit on top when the iPad is at a high angle. We also were able to ask him more specific questions about his habits around taking commissions that we weren’t able to ask during our initial meeting because we hadn’t fully settled on an idea to probe more into the behavior around it; for example, he told us that he works on around 5 at a time, and would like to have them labelled in the system, supporting our choice to shift from constant buttons assigned to three projects at once to a menu system.

It would have been helpful for us to have at least some of the circuitry and software finished so that we could see if he felt comfortable interacting with it, and able to understand the basic mechanisms and instructions to use it. While we showed him paper sketches of what the interface and surrounding hardware would most likely look like and how it would be used, and he showed no issues in understanding how it would work, it would have been helpful to have had some part ready for him to interact with. Often hearing how something will work is easy to understand, actually using said thing is often when problems arise.

Our next steps include deciding on a final form factor for the board and creating the electronic interface. The arm mechanism has been finalized at this point, so we need to either find or design a bracket to interface with the holder. Next, the electronic aspects and layout need to be finalized in order to decide a final form factor for the holder. Code has been written and compiled but not physically tested, due to not having built the circuitry that would allow it to operate, which we will be able to do once we receive the new display. Once the electronics are taken care of, the test board will be milled and the prototype assembled to verify that the form factor is satisfactory. We plan to meet with Richard before the final critique to make sure that he is happy with our functional prototype before we move to our final version. After this meeting, the final board and baseboard will be made and polished and the final product assembled.