We are Team MARYa, (Mohan, Andrea, Roly, Yingyang) a group of four Carnegie Mellon University students taking an Introduction to Physical Computing course. For our final project, we were given the task of pairing up with an older person, Maria and creating a useful device for her that implements an Arduino and other physical computing elements. At our initial meeting, we discovered Maria had a habit of throwing papers on the floor instead of organizing them. Keeping this in mind, we created a prototype for a shelf that would sort your papers for you. Be warned, our prototype is very different from our final product which is documented here. Due to some constraints, we had to take our project in a different direction.
What We Built:
This shelf is helping Maria to easily organise her documents in her home working space. Each section on the shelf has a category with a number and match to the button. The shelf section will be blink by pressing the button. It will be stopped blinking by receiving the document in the right shelf section.
Overall
Botton Layouts
Shelf Construction
OLYMPUS DIGITAL CAMERA
Maria is busy working in her home office. She wants to quickly organise her documents on her messy desk. After reading her latest bank invoice, she presses the button “3”, which identifies as “Finance” section. She saw the section 3 on the shelf starts to blink so she put the letter into the right section. All the action takes Maria less than 2 secs. Maria is so happy that she can quickly organise her document without thinking where is the “Finance” section. Maria starts to like to organise her documents and she can finally find her documents in the right category in her shelf!
How We Got Here:
As was previously mentioned, our project took a sudden turn near the end of the process. We realized that the mechanical aspects of our filing cabinet (including a small elevator to place papers in shelves) were proving far too finicky to finish on time.
Roly here is removing the old supports for the elevator mechanism after trying to solve mechanical issues.
Due to the nature of timing belts and how precise they need to be to produce the motion desired, the elevator was experiencing problems with keeping the belts tensioned enough. These belts were essential to the elevator’s movement since they provided the force that spun the lead screws, which in turn lifted or lowered the tray.
With this big change, the team needed to decide on an alternative option that kept the spirit of the idea. The first idea came almost right away. We wanted to help Maria make sure her papers got filed, but we opted to make it a device to help modify behavior rather than perform the task itself.
This was taken moments after we decided that we had to change our idea. Written on the table were some notes on the logistics of the new idea.
These LED’s became a central part of our new idea.
This is a sensor similar to the one we used to detect when paper was placed in a shelf. The clear bulb emits infrared light, and when an object bounces it back to the black bulb, the sensor knows how far the object is.
Our device was changed slightly to include LED lights on the side of the shelves. These lights would illuminate the shelf where Maria was supposed to put a piece of paper (she would first pick the filing category with a button). Inside of the shelf, some device would sense that she put the paper there and turn off the light. This version of the device was now more focused on reinforcing good habits of paper filing. By having Maria file her papers immediately, we were trying to break the habit of simply leaving it for later. But one problem was left: how to incentivize this.
This was our inspiration for an incentive: a gumball machine. Maria had wanted some sort of candy dispenser before, so we figured we could bring it back in a similar form to this.
Based on one of our first meetings with Maria, we decided to use candy as the incentive for filing. Maria had initially proposed that we make her an automatic candy dispenser, but we decided upon our current project since it seemed more useful to her. By bringing back the candy idea, we added a little inside humor between us and created a reward system for filing.
First test of the lighting and paper detection!
Once we got the shelf lighting and paper detection system working, we began designing the reward system. It was going to be a simple box on top of the shelf that contained a mechanism to dispense candy. Unfortunately, when we were going to laser cut the box, the USB that its file was on got corrupted and we lost it permanently.
In the end, we were only able to get the lighting and paper detection system working, and it was very finicky. We were slightly disappointed to the project turn out like this, but we simply did not have time to fix our ranging technical issues by the day of the presentation.
The final circuit board for the shelving unit. It was somewhat cramped, but it worked (mostly).
Throughout this process, we learned a lot about working on teams with vastly different skill sets. One of our members was an architect, one was a costume designer, and the other two were Information Systems students. We had to be very thorough in our meetings to ensure that this very widespread team was always on the same page. We also had to learn to keep frustrations in check, seeing as we were working hard and late up until the last minute.
Patience was key throughout this whole process. We were working very hard and had to make sure not to clash heads.
Overall, this project was a semi-success in that we were able to finish something for Maria. We would have been disappointed to not have anything at all to present to her or the class. However, through diligence and taking responsibility for our own sections of work, we pushed through to the end.
Conclusions and Lessons Learned:
Assemble process
From the problems we as a team encountered at final assblem stage, we learn a important lesson that for the problem we never worked before, there should be enough time set aside for it, different parts could be delivered late, for example the belt we had at last minute won’t fit the mechanism we already had. Also building a quick prototype from what we have might be a good idea to identify problems need to be resolved. If time constrain is a issue, different scenarios could be developed in advance.
Final Presentation feedback
We get a lot of useful feedback during crit that would help us identify problems and know better about outside opinion, overall people like the project. Although we change our idea at last minute, it is actually from the rewarding system idea we had long time ago. In terms of helping Maria on her habit, reminding system would potentially help her more on change her habbits, as she wrote in comment, it’s more a human/motivation issue than a mechanical issue, how we address it is very interesting, and how the client would use it is a very important post-evaluation part too. There are advice from students suggesting that visual reminder(blinking LED)might be substitued by audio which is more “annoying” and some people feel like bliking LED is not very comfortable to them, therefore this is a debating point which might need to discuss with our “client”. Right now the shelf is not painted and LED stripe is exposed, both need to be worked on in future.
Overall, our team learned a lot from this project. We worked closely with Maria trying to identify her needs in daily life that herself might even not aware of. We tried to start from the point of “organizing paper work” and develop a system/mechanism that would work to help her better on it. We stumbled on some issues and succeed on other issues, both are good lessons we learned.
Technical Details:
/*
* Lighted Paper Shelf by Andrea Chung
*
* When a button is pushed, its corresponding shelf section will
* light up and blink until a photocell sensor reads that something
* has been inserted into that shelf.
*
* The circuit:
* - 6 photocell sensors attached to pins A0-A5
* - 6 corresponding buttons attached to pins 2-7
* - LED strip attached to pin 8
*/
//photocell sensor pins
int sensor1 = A0;
int sensor2 = A1;
int sensor3 = A2;
int sensor4 = A3;
int sensor5 = A4;
int sensor6 = A5;
//button pins
const int BUT1 = 2;
const int BUT2 = 3;
const int BUT3 = 4;
const int BUT4 = 5;
const int BUT5 = 6;
const int BUT6 = 7;
//filtering values for the sensor readings
float avgVal = 0; // this variable, of type float (decimal), holds the filtered sensor value
const float PREV_WEIGHT = 0.8; // this variable, of type float (decimal), determines the previously filtered value's weight.
const float CUR_WEIGHT = 1 - PREV_WEIGHT;
const float threshold = 5.0;
//timer setup for blinking LEDs
unsigned long previousMillis = 0;
const long interval = 1000;
//indicattors for light on or off
bool on1 = false;
bool on2 = false;
bool on3 = false;
bool on4 = false;
bool on5 = false;
bool on6 = false;
//LED strip setup
#include <PololuLedStrip.h>
PololuLedStrip<8> ledStrip;
// set up how many LEDs on the strip you would like to control
#define LED_COUNT 48
rgb_color colors[LED_COUNT];
void setup() {
//setup sensor pins as input
pinMode(sensor1, INPUT);
pinMode(sensor2, INPUT);
pinMode(sensor3, INPUT);
pinMode(sensor4, INPUT);
pinMode(sensor5, INPUT);
pinMode(sensor6, INPUT);
//setup button pins as pullup
pinMode(BUT1, INPUT_PULLUP);
pinMode(BUT2, INPUT_PULLUP);
pinMode(BUT3, INPUT_PULLUP);
pinMode(BUT4, INPUT_PULLUP);
pinMode(BUT5, INPUT_PULLUP);
pinMode(BUT6, INPUT_PULLUP);
Serial.begin(9600); // starts serial communication at 9,600 baud (the rate)
}
void loop() {
//read button values
int but1Val = digitalRead(BUT1);
int but2Val = digitalRead(BUT2);
int but3Val = digitalRead(BUT3);
int but4Val = digitalRead(BUT4);
int but5Val = digitalRead(BUT5);
int but6Val = digitalRead(BUT6);
rgb_color color;
//current time passed
unsigned long currentMillis = millis();
//if button 1 pressed set as on and read sensor value
if (but1Val == LOW) {
on1 = true;
avgVal = (float)analogRead(sensor1);
Serial.println("button1");
}
//if LED is set as on make light blink until the sensor
//reading average changes suddenly
if (on1 == true){
//make led blink
if (currentMillis - previousMillis >= interval) {
// save the last time you blinked the LED
previousMillis = currentMillis;
// if the LED is off turn it on and vice-versa:
if (color.red > 0) {
color.red = 0;
color.green = 0;
color.blue = 0;
}
else {
color.red = 255;
color.green = 0;
color.blue = 0;
}
for (uint16_t i = 0; i < 8; i++)
{
colors[i] = color;
}
ledStrip.write(colors, LED_COUNT);
}
//equations taken from Phys Comp course website credit to Joseph Paetz
float cur_reading = (float)analogRead(sensor1);
avgVal = avgVal * PREV_WEIGHT + cur_reading * CUR_WEIGHT;
Serial.println(cur_reading);
//turn LED off and set as off if the reading changes over a set threshold
if (abs(avgVal - cur_reading) > threshold){
color.red = 0;
color.green = 0;
color.blue = 0;
for (uint16_t i = 0; i < 8; i++)
{
colors[i] = color;
}
ledStrip.write(colors, LED_COUNT);
on1 = false;
}
}
//if button 2 pressed set as on and read sensor value
if (but2Val == LOW) {
on2 = true;
avgVal = analogRead(sensor2);
Serial.println("button2");
}
if (on2 == true){
//make led blink
if (currentMillis - previousMillis >= interval) {
// save the last time you blinked the LED
previousMillis = currentMillis;
// if the LED is off turn it on and vice-versa:
if (color.red == 255) {
color.red = 0;
color.green = 0;
color.blue = 0;
}
else {
color.red = 255;
color.green = 255;
color.blue = 0;
}
for (uint16_t i = 8; i < 16; i++)
{
colors[i] = color;
}
ledStrip.write(colors, LED_COUNT);
}
float cur_reading = (float)analogRead(sensor2);
avgVal = avgVal * PREV_WEIGHT + cur_reading * CUR_WEIGHT;
Serial.println(cur_reading);
if (abs(avgVal - cur_reading) > threshold){
color.red = 0;
color.green = 0;
color.blue = 0;
for (uint16_t i = 8; i < 16; i++)
{
colors[i] = color;
}
ledStrip.write(colors, LED_COUNT);
on2 = false;
}
}
//if button 3 pressed set as on and read sensor value
if (but3Val == LOW) {
on3 = true;
avgVal = analogRead(sensor3);
Serial.println("button3");
}
if (on3 == true){
//make led blink
if (currentMillis - previousMillis >= interval) {
// save the last time you blinked the LED
previousMillis = currentMillis;
// if the LED is off turn it on and vice-versa:
if (color.red == 255) {
color.red = 0;
color.green = 0;
color.blue = 0;
}
else {
color.red = 255;
color.green = 255;
color.blue = 0;
}
for (uint16_t i = 16; i < 24; i++)
{
colors[i] = color;
}
ledStrip.write(colors, LED_COUNT);
}
float cur_reading = (float)analogRead(sensor3);
avgVal = avgVal * PREV_WEIGHT + cur_reading * CUR_WEIGHT;
Serial.println(cur_reading*10);
if (abs(avgVal - cur_reading) > threshold){
color.red = 0;
color.green = 0;
color.blue = 0;
for (uint16_t i = 16; i < 24; i++)
{
colors[i] = color;
}
ledStrip.write(colors, LED_COUNT);
on3 = false;
}
}
//if button 4 pressed set as on and read sensor value
if (but4Val == LOW) {
on4 = true;
avgVal = analogRead(sensor4);
Serial.println("button4");
}
if (on4 == true){
//make led blink
if (currentMillis - previousMillis >= interval) {
// save the last time you blinked the LED
previousMillis = currentMillis;
// if the LED is off turn it on and vice-versa:
if (color.blue == 255) {
color.red = 0;
color.green = 0;
color.blue = 0;
}
else {
color.red = 0;
color.green = 0;
color.blue = 255;
}
for (uint16_t i = 24; i < 32; i++)
{
colors[i] = color;
}
ledStrip.write(colors, LED_COUNT);
}
float cur_reading = (float)analogRead(sensor4);
avgVal = avgVal * PREV_WEIGHT + cur_reading * CUR_WEIGHT;
Serial.println(cur_reading);
if (abs(avgVal - cur_reading) > threshold){
color.red = 0;
color.green = 0;
color.blue = 0;
for (uint16_t i = 24; i < 32; i++)
{
colors[i] = color;
}
ledStrip.write(colors, LED_COUNT);
on4 = false;
}
}
//if button 5 pressed set as on and read sensor value
if (but5Val == LOW) {
on5 = true;
avgVal = analogRead(sensor5);
Serial.println("button5");
}
if (on5 == true){
//make led blink
if (currentMillis - previousMillis >= interval) {
// save the last time you blinked the LED
previousMillis = currentMillis;
// if the LED is off turn it on and vice-versa:
if (color.blue == 255) {
color.red = 0;
color.green = 0;
color.blue = 0;
}
else {
color.red = 0;
color.green = 0;
color.blue = 255;
}
for (uint16_t i = 32; i < 40; i++)
{
colors[i] = color;
}
ledStrip.write(colors, LED_COUNT);
}
float cur_reading = (float)analogRead(sensor5);
avgVal = avgVal * PREV_WEIGHT + cur_reading * CUR_WEIGHT;
Serial.println(cur_reading);
if (abs(avgVal - cur_reading) > threshold){
color.red = 0;
color.green = 0;
color.blue = 0;
for (uint16_t i = 32; i < 40; i++)
{
colors[i] = color;
}
ledStrip.write(colors, LED_COUNT);
on5 = false;
}
}
//if button 6 pressed set as on and read sensor value
if(but6Val == LOW) {
on6 = true;
avgVal = analogRead(sensor6);
Serial.println("button6");
}
if (on6 == true){
//make led blink
if (currentMillis - previousMillis >= interval) {
// save the last time you blinked the LED
previousMillis = currentMillis;
// if the LED is off turn it on and vice-versa:
if (color.blue == 255) {
color.red = 0;
color.green = 0;
color.blue = 0;
}
else {
color.red = 0;
color.green = 0;
color.blue = 255;
}
for (uint16_t i = 40; i < 48; i++)
{
colors[i] = color;
}
ledStrip.write(colors, LED_COUNT);
}
float cur_reading = (float)analogRead(sensor6);
avgVal = avgVal * PREV_WEIGHT + cur_reading * CUR_WEIGHT;
Serial.println(cur_reading);
if (abs(avgVal - cur_reading) > threshold){
color.red = 0;
color.green = 0;
color.blue = 0;
for (uint16_t i = 40; i < 48 ; i++)
{
colors[i] = color;
}
ledStrip.write(colors, LED_COUNT);
on6 = false;
}
}
else{
color.red = 0;
color.green = 0;
color.blue = 0;
for (uint16_t i = 40; i < 48 ; i++)
{
colors[i] = color;
}
ledStrip.write(colors, LED_COUNT);
}
}
Schematic
For the final project of Introduction to Physical Computing, The Weavers, which consists of Jenny, Megan, and Ghalya, were assigned to create an assistive device for Rebecca Herbert. Rebecca loves to weave and the most tedious task in weaving is winding the yarn into a ball from its original loose packaging. Not only does this tedious hurt her wrist, but she also has to estimate the amount of yarn that she has. This documents the process of creating a motorized ball winder that would save Rebecca from doing this task manually.
Link to Prototype Documentation: https://courses.ideate.cmu.edu/60-223/f2018/work/team-weavers-prototype-documentation/
Link to Initial Interview: https://courses.ideate.cmu.edu/60-223/f2018/work/interview-documentation/
What We Built
Details of Ball Holder
The Automatic Ball Winder is composed of three main components. First, the screen and buttons allow the user to interact with the device and input yards of yarn needed as well as emergency stop, reset, and continue buttons that let the user change his/her mind. Next is a distance encoder. The user feeds the yarn through metal feeder loops and around a disk. The disk measures distance by counting the number of rotations while the metal feeder loops keep the yarn at the right tension and height so it doesn’t fall off. When the user presses “start”, the motor that is attached to the yarn holder will spin, spinning both the encoder as well as winding yarn onto the holder. If the yarn falls off or encounters a knot, the device will automatically shut off to reduce error.
Ball Winder in Action with Yarn Umbrella.
Imagine Rebecca wants to start a new weaving project, but her hands are hurting too much to use the manual ballwinder, and since she’s working with silk she can’t afford to ball up too much more than what is necessary. She goes to her Automatic Ballwinder, and presses the “reset” button, which makes it say “Welcome Rebecca”, and “Yarn to wind:”. She turns the knob until she gets to the desired amount to wind, presses start, and goes to make herself some tea.
After about 15 minutes, she comes back and sees that the ballwinder is done, and she screws off the top so she can use the yarn there. She still needs some more for another project, so she prepares another skein and heads off to start her project.
Threading Yarn Through Distance Encoder
Putting String onto Ball Holder
Unscrewing Ball Holder
Ball Holder Unscrewed
Details of Distance Encoder and Buttons
Wire Thread Feeders
How We Got Here
During the initial meeting with Rebecca, it was clear that she loved to weave. She vocalized how strongly she disliked the ball winding process and how commercial ball winders were upwards of 300 dollars. Since that meeting, Team Weavers decided the product that would best fit her lifestyle would be an automatic ball winder that would tell her distance of yarn wound and save her wrists from carpal tunnel.
Initially, the main thing that we focused on was the criss-cross yarn pattern that the handheld ballwinder created. With the winding method of the motor, the pattern was mostly parallel. After a few days, we decided and Rebecca assured us that the yarn pattern was not as important as the distance encoding and the automatic function of the winding. At the end of the project, we found that consistent tension in the yarn and speed without erratic stop and start would create a decent yarn pattern that would not entangle.
Manual Ball WInder Yarn Pattern
Prototype Yarn Pattern
Final Yarn Pattern: Consistent Tension and Speed
To get an accurate distance reading for the yarn, we created a disk that the yarn wrapped around. When the motor for the yarn holder spun, it would pull on the yarn around the disk and turn the encoding disk. Counting the number of turns would give an accurate reading of yarn wound on the ball. Some of the problems we encountered with the encoder were that the yarn kept falling out of the disk, making the reading inaccurate. The ball winder would spin and pull yarn onto the ball, but the encoder would not spin. This was a big problem because not only would the device not know that the yarn was being added and therefore stop, but it would deceive Rebecca on how much yarn was wound onto the ball.
Testing the Length of String After Finished.
To deal with this problem, we turned to both software and the physical form of the device. In terms of software, we decided that if the motor was running and the encoder was not, then the yarn had fallen off and to switch to an “emergency stop” mode. During this mode, the user can read how much yarn is currently wound as well as continue the winding process or restart. One of the physical changes was to make the shape of the encoder have a larger lip and therefore less chance to fall out. The second preventative physical measure was to create wire guides that served as a way to feed the yarn at the right height as well as create the right pull tension so that the yarn would not fall out. Also, the wire guides instead of the prototype wood guards make it easy to remove the yarn without being at the end of the string.
Initial Prototype of Distance Encoder
Second Iteration of Distance Encoder: Lowered yarn holders to prevent yarn from slipping out.
Final Distance Encoder: Metal wires allow for easy usage and larger gap in the disk prevents slipping.
Finally, instead of an overall level box for the shape of the device, we decided to create levels. After much testing, we found that having the encoder be at an elevated surface that feeds the string directly parallel to the yarn winder would increase the tension in the string and encourage the string not to be found on the bottom of the yarn holder or fall off the track.
Original Top for the device.
Second Iteration of Box
Moving from the prototype to the finished device had some aesthetic and physical challenges as well. We ended up choosing white acrylic for the overall form of the box and white plastic to create the yarn holder to mimic the original feeling of the handheld ball winder. Also, the screen updates the user as to how many yards have been wound, if the emergency stop was triggered, and if the winding has been completed.
Original Wood Prototype
Prototype Ball Winder with Ball
Megan Working on Soldering the Final Board
We spent awhile soldering the components in, to make the project more permanent and portable; in addition, we glued the box shut for easy portability.
Digital Sketch of Final Device Structure
For the above design, we went with white acrylic to make it seem more attractive and presentable, and we used the teeth to make our box easier to assemble and sturdier. We added some space for buttons, and a port in the back for Rebecca to plug in her winder.
Making of Acrylic Box
The laser cutter did not fully cut through the acrylic, so we had to try and rip a lot of pieces out- though it was difficult, we wound up getting most to come out and by using only a little extra acrylic and a filer, we were able to get the box made.
Welcome Screen
Winding Screen
We created a few “states” for the screen- two are shown above- one for the intro, where we welcome Rebecca, one where we ask how much she wants wound, one that shows how much has been wound, one if our winder emergency stops, and one for when it finishes.
Conclusions and Lessons Learned
The most rewarding thing about this project was the final critique. When we showed the project to Rebecca, something that she has been wanting for quite some time, she was so excited to bring the device home to use. The biggest motivator to making sure this device was working flawlessly was knowing that Rebecca would be using this daily and it would make such a positive impact in her life. Even during the creation process, many people who also had interests in fabrics and weaving reaffirmed the benefit of this device to us. We were surprised how much this device meant to her/other weavers and it was satisfying to create a project that had a purpose.
We also gained some important insights from others during the final critique process. One person suggested to “[have] a flap to keep the yarn on the disk” which would make it hard for the string to fall out as well. To fix this problem, Rebecca suggested we use a piece of tape would be an easy way to incorporate this idea. Another suggestion was to have an “adjustable speed, so customers can change the tightness themselves”. This would be an interesting component to add in case different types of yarn break under different pressures. Thus, having an adjustable speed or tension setting would account for more types of fibers.
Had we had more time to perfect the device, we would have tested the length of string over 600 yards. Even though Rebecca typically uses between 250-450 yards of string, we want to make sure that no matter how long the string is that the reading is accurate. During this entire process, one of the most educational aspects was realizing to test components as we go. After having the device begin to smoke up after plugging in power to our initial final product with a sultered final board, we had to go back to our board and take apart each component to isolate the problem. We found that testing each piece as we went was a much easier way to deal with this problem in the future.
Final Presentation with Rebecca.
Technical details
Code
/*
Automatic Ball Winder
Jenny Han
Ghalya Alsanea
Megan Roche
The following code uses an Arduino, stepper moter, two rotary encoders, buttons
and LCD Display.
It maintains four states: Welcome, Winding, Emergency Stop, and Finished.
Welcome allows users to input the amount of yarn that the user needs.
Winding starts the motor to collect the yarn into a ball as well as
counting amountof yarn wound.
Emergency Stop detects if the yarn fell off and displays how much
yarn was wound at that instant.
Finished stops the motor when the distance wanted is achieved.
*/
#include <Encoder.h>
#include <AccelStepper.h>
#include <Wire.h>
#include <LiquidCrystal_I2C.h>
LiquidCrystal_I2C lcd(0x27, 16, 2);
unsigned long timer = 0;
long int stepsToDo = 0;
//output rotary encoder: distance measuring
const int O_CLK = 2;
const int O_DT = 3;
const int O_SW = 4;
//input rotary encoder: distance input
const int I_CLK = 5;
const int I_DT = 6;
const int I_SW = 7;
//buttons for emergency stop and continue
const int ESTOP = 11;
const int CONTB = 12;
long val;
//current state that the device is in
bool stateWelcome = false;
bool stateWind = false;
bool stateStop = false;
bool stateFinished = false;
bool winding = false;
//variables to keep track of yarn wound
long oldPosition = -999;
long OLDLENGTH = -1;
long NEWLENGTH;
int CURRENTD;
long TOTALD = 0;
long int finalLen = 0;
//stepper motor pins
const int STEP_PIN = 8;
const int DIR_PIN = 9;
AccelStepper winderMotor(1, STEP_PIN, DIR_PIN);
//initialize encoders
Encoder encLength(O_CLK, O_DT);
Encoder encInput(I_CLK, I_DT);
void setup() {
lcd.init();
//Serial.begin(9600);
lcd.backlight();
lcd.setCursor(0, 0);
lcd.print("Welcome Rebecca!");
delay(4000);
lcd.clear();
pinMode(O_SW, INPUT);
pinMode(I_SW, INPUT);
winderMotor.setMaxSpeed(1000);
winderMotor.setSpeed(400);
pinMode(ESTOP, INPUT_PULLUP);
pinMode(CONTB, INPUT_PULLUP);
stateWelcome = true;
}
void loop() {
//run welcome function
if (stateWelcome) {
stateWelcomeFunction();
}
//run winding
if (stateWind) {
stateWindFunction();
}
//run emergency stop
if (stateStop) {
stateStopFunction();
}
//run finished screen
if (stateFinished) {
stateFinishedFunction();
}
}
//takes in distance desired
void stateWelcomeFunction() {
long newPosition = encInput.read();
if (val < 0) {
val = 0;
}
//starts at 350, average yarn length needed
if (newPosition != oldPosition) {
oldPosition = newPosition;
val = 350 - 5* (newPosition/4);
lcd.clear();
lcd.leftToRight();
lcd.setCursor(0, 0);
lcd.print("Yards needed:");
lcd.setCursor(0, 1);
lcd.print(val);
}
//cannot go negative
if (!digitalRead(I_SW)) {
TOTALD = val;
lcd.clear();
lcd.leftToRight();
lcd.setCursor(0, 0);
lcd.print("Winding ");
lcd.setCursor(0, 1);
lcd.print(TOTALD);
lcd.print(" Yards.");
delay(2000);
stateWelcome = false;
stateWind = true;
}
}
//starts winding motor and counts distance
//checks if yarn fell off
void stateWindFunction() {
int rotNeeded = TOTALD * 6;
int oldD = rotNeeded / 6;
while (rotNeeded > 0) {
//if stop pressed then go to state stop function
bool stateStopButton = digitalRead(ESTOP);
if (stateStopButton == 0) {
stateWind = false;
stateStop = true;
break;
}
//every four seconds check if string fell off
if (millis() - timer >= 4000 ) {
// this happens once per 4 seconds
NEWLENGTH = encLength.read();
if ((NEWLENGTH == OLDLENGTH) && (NEWLENGTH != 0)) {
stateWind = false;
stateStop = true;
break;
}
timer = millis();
}
NEWLENGTH = encLength.read();
//calls for motor to run
for (int i = 0; i < 700; i++) {
winderMotor.runSpeed();
}
//update LCD screen
if ((NEWLENGTH != OLDLENGTH)) {
OLDLENGTH = NEWLENGTH;
rotNeeded = (TOTALD * 6) - floor(abs(NEWLENGTH / 800));
CURRENTD = rotNeeded / 6;
if (CURRENTD != oldD) {
oldD = CURRENTD;
lcd.clear();
lcd.setCursor(0, 0);
lcd.print("TO GO : ");
lcd.setCursor(0, 1);
lcd.print(CURRENTD );
lcd.print(" yards");
}
if (rotNeeded == 0) {
stateWind = false;
stateFinished = true;
}
}
}
}
//finished state, restart button is the reset pin
void stateFinishedFunction() {
//if button press
lcd.clear();
lcd.setCursor(0, 0);
lcd.print("Finished!");
lcd.setCursor(0, 1);
lcd.print("Wound ");
lcd.print(TOTALD);
lcd.print("yds");
}
//emergecy stop state, restart button is the reset pin
void stateStopFunction() {
lcd.clear();
lcd.setCursor(0, 0);
lcd.print("STOPPED!");
lcd.setCursor(0, 1);
lcd.print("Wound ");
lcd.print(TOTALD - CURRENTD - 1);
lcd.print("yds");
int contbutton = digitalRead(CONTB);
//continue
if (contbutton == 0) {
delay(100);
stateStop = false;
stateWind = true;
stateFinished=false;
stateWelcome = false;
}
}
Schematic and design files
]]>
For this project, our group of three were tasked with creating a device that would help make the life of our older friend, Jim (as he prefers to go by), better. Though we’re limited by what we’ve learned so far, we were excited to face this task on and to help make something that would survive past the end of this class that would benefit someone else. From our initial first meeting, it is clear to see that the device that would benefit Jim the most would be a device to make sure he doesn’t forget what he need when he leaves the house. We also have another post highlighting what occurred during the first meeting here as well as this post of where our prototype was at about three weeks ago.
WHAT WE BUILT
Description
Our project is a box that is to be installed by Jim’s front door that contains a screen that will display Jim’s items of the day and a motion sensor that triggers the device when he approaches the door. The Google API parses through Jim’s Google Calendar to identify a list of common events or take a list of items from the description of non-common events to display in a list format on his door. Whenever he walks approximately 3 meters by the door, the motion sensor will trip, turning on the display that will remind him to make sure he has all the items he needs for the day.
Final Photographs
Overall Content Photo
Standard opening screen
Barrel jack power and ground connection at the base of the box with motion sensor clearly visible
Temporary velcro closing mechanism
Items list screen(1/3)
Keyboard screen (2/3)
Goodbye screen (1/3)
Narrated Sketch of the Intended Use
Every morning, Jim wakes up to a busy day filled with community activities and long board meetings. Those days often require Jim to bring with him many important supplies, without which his day is not as productive. Often, Jim makes note of the things he needs in his Google Calendar, but that is often in a place that is not as easily accessible to him when he is rushing out the door. Imagine this scenario: Jim is walking out the door in the morning, hoping he remembers everything he needs for the day. Just then, a motion detector by the door detects his presence, and, in response, an LED touchscreen lights up, asking Jim if he has everything he needs for the day based on the entries in his calendar. If the answer is yes, then Jim touches the on-screen button to indicate as such, and off he goes, day’s supplies in tow. This is Jim’s daily reminder box helping make his life easier and more productive, one notification at a time.
HOW WE GOT HERE
Our original idea was to have the ESP connect to a server that parses data from Google API to interpret Jim’s calendar for the day. Then the items for those events would be communicated back to the ESP to be displayed in the form of a checklist on the TFT Display as a reminder when he leaves the house.
Sketch of one of the earliest planning diagrams when the Arduino was incorporated
Decided not to set up server. We ended up getting recommended to avoid setting up a server, however, because it would have been really difficult and cost money to maintain. Instead we redesigned the system to parse all data locally and on-premise, which had the trade-off of taking up significantly more memory on the ESP32, out IoT chip.
Ordering a second TFT Display. The next part of the process was to adapt the Adafruit ILI9341 library to do exactly what we needed on the touchscreen display. After the prototype we ordered a slightly bigger screen based off the request of Jim. This served to be useful in the long run since it allowed two people to work n the screens simultaneously and independently. Catherine’s biggest problem was getting the touch to respond properly either due to wearing down the screen or something wrong with the smaller screen.
Attempt#1 for coordinates and basic code with display
While she tinkered with the bigger screen, Lexi experimented with displaying icons to improve visibility of Jim’s items.
Updating the screen based on input from the Serial Monitor
Initially and for demo purposes, we set up the touchscreen to behave like the Google API by pulling data from the Serial monitor. When the screen would sense something written into the Serial monitor, it would perform a Serial.read() to pull that data and print it on the touchscreen.
Serial and display communicating
Getting a keyboard on the display through the ESP. We discovered that there were many problems implementing touch onto the TFT display when connected to the ESP.
Tft display with touchscreen keyboard
We needed to have the keyboard so that Jim could input the verification required to access his wifi once he moved houses. This is because the ESP will not be able to communicate with anything else in the system until it is connected to the WiFi. Ultimately we decided that it’d be better to prioritize the touchscreen keyboard and connected the display to the Arduino which would in turn communicate with the ESP through the RX and TX pins. This also resulted in the scrapping of the icons due the example pictureEmbeded only working with an ESP.
Icon conversion issues. The first problem with the icon displaying was that it took Lexi a week to realize the example code, Adafruit ILI9341’s pictureEmbeded, wasn’t designed to work on Arduino Unos, which she had been testing with. This code doesn’t work with an Arduino, however, so we had to scrap it. Even before we scrapped it though, we were running into the issue that the image converter Lexi was using to the transfer the pixel measurements of the icon into hex codes kept returning values of at least 15,500 values. uint16_t is able to store at most 14,000 values and experimenting with image sizing and various settings would either increase the number of values or have it stay the same. We ended up ditching the entire idea before she had to attempt to find another image converting method.
Image converted into hex codes
Router to connect ESP to Wifi. It would have been useful to discover earlier that a router could be set up to bypass the issue of the ESP connecting to CMU-SECURE. This is due to the way the ESP handles the wifi verification. Since it is much harder to connect to WEP-Enterprise networks, testing anything on campus was impossible. It would have been nice to realize that a router was an option a few days before we did. At that point, there had already been a few days of Catherine testing various bits of code that Sana wrote and implementing at Catherine’s house due to Sana using the school wifi at home.
Memory Problems with the ESP. The ESP32, despite being a chip that can access the internet, holds a very small amount of memory. Accessing the Google Calendar API returned an extremely long string of characters, which was large enough to cause overflow in memory. As a result, the ESP32 turned out being the wrong device for us to use for this project.
Soldering all the wires together to make the connections inside the box more durable
Testing the power and ground side of the DC Jack using the multimeter
Velcro-ing the box closed. Originally we were thinking about gluing the box closed with everything inside it.
The box post-soldering pre-velcroing
Due to the impermanent nature of our box at the time of the demo, we decided to velcro the box closed. This was because velcro is the easiest way of closing the box without prior planning and worked pretty well for what we needed. We realized by doing this that it could actually benefit the box for it to have the option of opening in case anything goes wrong/something gets jostled when Jim moves between houses. As a result, we decided to add latches and a hinge to attach to the final one but didn’t manage to get them in time for documentation.
Velcro-sealed box
CONCLUSIONS
The final crit proved to be a very helpful time for us to learn about good features to add to the project. Overall response to the box itself was very positive- both from older friends and from peers. Overall, it seemed as if both groups were very willing to use the box in daily life. Some feedback was that it may have been better to change the design of the box so that it draws a little more attention to itself- like a coat of varnish and maybe a blinking LED light and some sound (these were all suggestions that came from 3 separate people). While we did what our client wanted (these were all ideas we discussed with him), they are certainly ideas we would have implemented.
Additionally, the idea of the demo in its simple form- without the Google calendar implementation- also garnered a good amount of support. It appeared to be a good solution for those who didn’t use a google calendar, though of course there was also the opposing side of the same argument- that the keyboard was too small and the Google Calendar would have worked much better. This reasoning, however, often came from the general sentiment that the screen itself was too small. We’d already bought the largest, reasonably priced screen that we could find, and going any bigger would have given us marginal improvements of size with dramatic price increases. By the end of this discussion, there was a mention that this product would have been great on an iPad, which then begged the question- why not mount an iPad by the door instead? Why this project? To us, this is because an iPad is a one-size-fits-all, expensive solution, whereas our box is a this-size-fits-Jim, more cost-effective solution.
Overall, this project is a quite simple idea, but, under the hood, the complexities of this project were so great that factors like time, WiFi access, and constraints in memory played a big role in making this project much harder to realize. While we were able to put together a final product that satisfied the needs of our client, we would have been much happier with a product that would have been able to access WiFi, a circumstance we couldn’t control. Testing and overall product development were severely impedimented by this, despite the fact that most of the code was written and only needed to be tested. Everything is currently in progress and will soon be updated to meet our standards of functionality, as well as Jim’s. Looking forward, one thing to change would be to ensure that the WiFi chip we are using has much more memory and is able to effectively access the internet from where it is being tested, and to pre-check that the devices being used are always compatible. Compatibility played a large role in the effectiveness of this project. Because IoT projects require hardware, connectivity, and software to work together seamlessly, the presence of many roadblocks in this path made a final product much more difficult to develop with the resources on hand. Still, the learning curve, though steep, was important and taught us a lot about the importance of developing products that are viable and useful. With a larger timeframe, however, we are fully confident that we can produce a fully functioning reminder box for Jim by the time spring semester rolls around.
TECHNICAL DETAILS
Code
The following code is a completely locally-hosted, on-premise code block operating only with the Arduino Uno R3, which takes entries based on the next day to add to the calendar:
/* this code is written for the Arduino component of the reminder box, and is the demo code,
* which allows the user to manually enter information about the things he/she needs for the
* next day. The reminder box pulls information on the day of the week (set based on an example
* schedule here) and prompts the user (whenever the motion sensor is tripped) to check off
* whether they have everything or if they want to add anything to the list of things they need
* for tomorrow.
*/
//libraries
#include <Adafruit_GFX.h>
#include <SPI.h>
#include <Wire.h>
#include <Adafruit_ILI9341.h>
#include <TouchScreen.h>
#include <stdint.h>
#include <DS3231.h>
//Touchscreen pins
#define YP A2
#define XM A3
#define YM 8
#define XP 9
//Calibration data
#define TS_MINX 150
#define TS_MINY 120
#define TS_MAXX 920
#define TS_MAXY 940
#define MINPRESSURE 10
#define MAXPRESSURE 1000
#define IsWithin(x, a, b) ((x>=a)&&(x<=b))
TouchScreen ts = TouchScreen(XP, YP, XM, YM, 600);
#define TFT_CS 10
#define TFT_DC 9
Adafruit_ILI9341 tft = Adafruit_ILI9341(TFT_CS, TFT_DC);
DS3231 rtc(SDA, SCL);
String dailyStuff[7] = {"\n agenda\n coat\n umbrella",
"\n wallet\n documents",
"\n gym bag\n dog treats",
"\n files\n pen drive",
"\n cell phone\n passport",
"\n car keys\n bags",
"\n art supplies\n laptop charger"};
// define variables for if he has everything and the new and old strings
bool hasEverything = false;
String old = "";
String updated = "";
String printed = updated;
String add = "";
String tomorrowString = "";
String tb = "";
int day;
//define LED and motion sensor pins
const int MOTION_SENSOR = 3;
const int LED = 4;
// set boolean for startup
bool remind = false;
int index = 0;
int bx;
int by;
bool scrollUp;
bool scrollDown;
bool adding;
const char textLimit = 25;
char MyBuffer[textLimit];
#define FRAME_X 210
#define FRAME_Y 180
#define FRAME_W 50
#define FRAME_H 40
#define GREENBUTTON_X (FRAME_X + FRAME_W)
#define GREENBUTTON_Y FRAME_Y
#define GREENBUTTON_W FRAME_W
#define GREENBUTTON_H FRAME_H
void drawFrame() {
tft.drawRect(GREENBUTTON_X, GREENBUTTON_Y + 7, FRAME_W, FRAME_H, ILI9341_BLACK);
}
void addMode() {
keyboard();
while (adding) {
GetKeyPress(MyBuffer);
}
yesBtn();
}
// updates the string needed for the day based on the time
void dayUpdate() {
if (rtc.getTime().hour == 23 && rtc.getTime().min == 58) {
tb = tomorrowString;
day = rtc.getTime().dow;
updated = dailyStuff[day] + tb;
tomorrowString = "";
}
}
void checkScroll() {
TSPoint p = ts.getPoint();
bx = map(p.y, TS_MINX, TS_MAXX, 0, tft.width());
by = map(p.x, TS_MINY, TS_MAXY, tft.height(),0);
if ((260 < bx && bx < 300) && (65 < by && by < 100)) {
scrollUp = true;
}
else if ((260 < bx && bx < 300) && (100 < by && by < 135)) {
scrollDown = true;
}
if (scrollDown) {
index = updated.indexOf("\n ");
if (index == -1) {
printed = updated.substring(index + 2);
tft.println(printed);
}
scrollDown = false;
}
if (scrollUp) {
index = updated.indexOf(printed);
if (index != 0) {
printed = updated.lastIndexOf("\n ", index);
tft.println(printed);
}
scrollUp = false;
}
}
//arrays for keyboard
const char Mobile_KB[3][13] PROGMEM = {
{0, 13, 10, 'Q', 'W', 'E', 'R', 'T', 'Y', 'U', 'I', 'O', 'P'},
{1, 12, 9, 'A', 'S', 'D', 'F', 'G', 'H', 'J', 'K', 'L'},
{3, 10, 7, 'Z', 'X', 'C', 'V', 'B', 'N', 'M'},
};
const char Mobile_NumKeys[3][13] PROGMEM = {
{0, 13, 10, '1', '2', '3', '4', '5', '6', '7', '8', '9', '0'},
{0, 13, 10, '-', '/', ':', ';', '(', ')', '$', '&', '@', '"'},
{5, 8, 5, '.', '\,', '?', '!', '\''}
};
const char Mobile_SymKeys[3][13] PROGMEM = {
{0, 13, 10, '[', ']', '{', '}', '#', '%', '^', '*', '+', '='},
{4, 9, 6, '_', '\\', '|', '~', '<', '>'}, //4
{5, 8, 5, '.', '\,', '?', '!', '\''}
};
//function that actually makes the keyboard look like a keyboard (has buttons and prints letters)
void MakeKB_Button(const char type[][13])
{
tft.setTextSize(2);
tft.setTextColor(ILI9341_WHITE);
for (int y = 0; y < 3; y++)
{
int ShiftRight = 15 * pgm_read_byte(&(type[y][0]));
for (int x = 3; x < 13; x++)
{
if (x >= pgm_read_byte(&(type[y][1]))) break;
drawButton(15 + (30 * (x - 3)) + ShiftRight, 100 + (30 * y), 20, 25); // this will draw the button on the screen by so many pixels
tft.setCursor(20 + (30 * (x - 3)) + ShiftRight, 105 + (30 * y));
tft.print(char(pgm_read_byte(&(type[y][x]))));
}
}
//ShiftKey
drawButton(15, 160, 35, 25);
tft.setCursor(27, 168);
tft.print('^');
//Special Characters
drawButton(15, 190, 35, 25);
tft.setCursor(21, 195);
tft.print(F("SP"));
//BackSpace
drawButton(270, 160, 35, 25);
tft.setCursor(276, 165);
tft.print(F("BS"));
//Return
drawButton(270, 190, 35, 25);
tft.setCursor(276, 195);
tft.print(F("RT"));
//Spacebar
drawButton(60, 190, 200, 25);
tft.setCursor(105, 195);
tft.print(F("SPACE BAR"));
}
//function that draws buttons
void drawButton(int x, int y, int w, int h)
{
// grey
tft.fillRoundRect(x - 3, y + 3, w, h, 3, ILI9341_LIGHTGREY); //Button Shading
// white
tft.fillRoundRect(x, y, w, h, 3, ILI9341_WHITE);// outter button color
//red
tft.fillRoundRect(x + 1, y + 1, w - 1 * 2, h - 1 * 2, 3, ILI9341_RED); //inner button color
}
void GetKeyPress(char * textBuffer)
{
static bool shift = false, special = false, back = false, lastSp = false, lastSh = false;
static int bufIndex = 0;
TSPoint p = ts.getPoint();
Serial.print("bx = "); Serial.print(bx); Serial.print(" by = "); Serial.println(by);
Serial.println(p.z);
Serial.println("we're here");
if (p.z >= 0)
{
//ShiftKey
if (TouchButton(-10, 160, 30, 25))
{
shift = !shift;
delay(200);
}
//Special Characters
if (TouchButton(-10, 190, 30, 25))
{
special = !special;
delay(200);
}
if (special != lastSp || shift != lastSh) //Start of changing the keys and make it look different
{
if (special)
{
if (shift)
{
tft.fillScreen(ILI9341_BLUE);
MakeKB_Button(Mobile_SymKeys);
}
else
{
tft.fillScreen(ILI9341_BLUE);
MakeKB_Button(Mobile_NumKeys);
}
}
else
{
tft.fillScreen(ILI9341_BLUE);
MakeKB_Button(Mobile_KB);
tft.setTextColor(0xffff, 0xf800);
}
if (special)
tft.setTextColor(0x0FF0, 0xf800);
else
tft.setTextColor(0xFFFF, 0xf800);
tft.setCursor(21, 195);
tft.print(F("SP"));
if (shift)
tft.setTextColor(0x0FF0, 0xf800);
else
tft.setTextColor(0xffff, 0xf800);
tft.setCursor(27, 168);
tft.print('^');
lastSh = shift;
lastSp = special;
lastSh = shift;
}//End of changing the keys and make it look different
for (int y = 0; y < 3; y++)
{
int ShiftRight;
if (special)
{
if (shift) //special+shift = symbol
ShiftRight = 15 * pgm_read_byte(&(Mobile_SymKeys[y][0]));
else //special only = number
ShiftRight = 15 * pgm_read_byte(&(Mobile_NumKeys[y][0]));
}
else //alphabet
ShiftRight = 15 * pgm_read_byte(&(Mobile_KB[y][0]));
for (int x = 3; x < 13; x++)
{
if (x >= (special ? (shift ? pgm_read_byte(&(Mobile_SymKeys[y][1])) : pgm_read_byte(&(Mobile_NumKeys[y][1]))) : pgm_read_byte(&(Mobile_KB[y][1])) )) break;
if (TouchButton((33 * (x - 3)) + ShiftRight - 14, 100 + (30 * y), 20, 25)) // this will draw the button on the screen by so many pixels
{
if (bufIndex < (textLimit - 1))
{
delay(200);
if (special)
{
if (shift)
textBuffer[bufIndex] = pgm_read_byte(&(Mobile_SymKeys[y][x]));
else
textBuffer[bufIndex] = pgm_read_byte(&(Mobile_NumKeys[y][x]));
}
else
textBuffer[bufIndex] = (pgm_read_byte(&(Mobile_KB[y][x])) + (shift ? 0 : ('a' - 'A')));
bufIndex++;
}
break;
}
}
}
//Spacebar
if (TouchButton(35, 190, 200, 25))
{
textBuffer[bufIndex++] = ' ';
delay(200);
}
//BackSpace
if (TouchButton(268, 160, 30, 25))
{
if ((bufIndex) > 0)
bufIndex--;
textBuffer[bufIndex] = 0;
tft.setTextColor(0, ILI9341_BLUE);
tft.setCursor(15, 80);
tft.print(F(" "));
delay(200);
}
//Return
if (TouchButton(268, 190, 30, 25))
{
tomorrowString = tomorrowString + textBuffer + "\n ";
Serial.println(textBuffer);
Serial.println("we got here too");
adding = false;
while (bufIndex > 0)
{
bufIndex--;
textBuffer[bufIndex] = 0;
}
tft.setTextColor(0, ILI9341_BLUE);
tft.setCursor(15, 80);
tft.print(F(" "));
Serial.println(tomorrowString);
yesBtn();
}
}
tft.setTextColor(0xffff, 0xf800);
tft.setCursor(15, 80);
tft.print(textBuffer);
}
//Makes you know if the point that you touched is in the range computed by x,y,w,h
byte TouchButton(int x, int y, int w, int h)
{
int X, Y;
// Retrieve a point
TSPoint p = ts.getPoint();
Y = map(p.x, TS_MINY, TS_MAXY, tft.height(), 0);
X = map(p.y, TS_MINX, TS_MAXX, 0, tft.width());
return (IsWithin(X, x, x + w) & IsWithin(Y, y, y + h));
}
void keyboard() {
tft.fillScreen(ILI9341_BLUE);
MakeKB_Button(Mobile_KB);
GetKeyPress(MyBuffer);
}
void yesBtn()
{
tft.fillScreen(ILI9341_BLUE);
//Asking Jim text
tft.setCursor(10, 10);
tft.setTextColor(ILI9341_WHITE);
tft.setTextSize(2);
tft.println("Hey Jim! Do you have what you need for today?");
//Printing item list
tft.setTextSize(2);
tft.println(updated);
//Scroll buttons
tft.fillRect(GREENBUTTON_X + 6, 60, GREENBUTTON_W - 12, 70, ILI9341_LIGHTGREY);
tft.drawRect(GREENBUTTON_X + 6, 60, GREENBUTTON_W - 12, 35, ILI9341_BLACK);
tft.drawRect(GREENBUTTON_X + 6, 95, GREENBUTTON_W - 12, 35, ILI9341_BLACK);
//Scroll up arrow
tft.setCursor(GREENBUTTON_X + 17, 65);
tft.setTextColor(ILI9341_BLACK);
tft.setTextSize(3);
tft.println((char)24);
//Scroll down arrow
tft.setCursor(GREENBUTTON_X + 17, 100);
tft.setTextColor(ILI9341_BLACK);
tft.setTextSize(3);
tft.println((char)25);
//Add item button
tft.fillRect(GREENBUTTON_X, 138, GREENBUTTON_W, GREENBUTTON_H, ILI9341_DARKGREEN);
tft.drawRect(GREENBUTTON_X, 138, GREENBUTTON_W, GREENBUTTON_H, ILI9341_BLACK);
tft.setCursor(GREENBUTTON_X + 7, 150);
tft.setTextColor(ILI9341_WHITE);
tft.setTextSize(2);
tft.println("ADD");
//YES button
tft.fillRect(GREENBUTTON_X, GREENBUTTON_Y + 7, GREENBUTTON_W, GREENBUTTON_H, ILI9341_RED);
drawFrame();
tft.setCursor(GREENBUTTON_X + 7 , GREENBUTTON_Y + (GREENBUTTON_H / 2));
tft.setTextColor(ILI9341_WHITE);
tft.setTextSize(2);
tft.println("YES");
}
void byeBtn()
{
//Have a good day text
tft.fillScreen(ILI9341_BLUE);
tft.setCursor(10, 10);
tft.setTextColor(ILI9341_WHITE);
tft.setTextSize(4);
tft.println("Great! Have a good day");
//bye button
tft.fillRect(GREENBUTTON_X, GREENBUTTON_Y + 7, GREENBUTTON_W, GREENBUTTON_H, ILI9341_GREEN);
drawFrame();
tft.setCursor(GREENBUTTON_X + 7 , GREENBUTTON_Y + (GREENBUTTON_H / 2));
tft.setTextColor(ILI9341_WHITE);
tft.setTextSize(2);
tft.println("BYE");
}
void setup() {
// Set up serial monitor
Serial.begin(9600);
pinMode(MOTION_SENSOR, INPUT);
pinMode(LED, OUTPUT);
tft.begin();
rtc.begin();
// Set up the screen at first
tft.setRotation(1);
yesBtn();
}
void loop() {
dayUpdate();
if (digitalRead(MOTION_SENSOR) == HIGH) {
digitalWrite(LED, HIGH);
//Serial.println("motion sensor triggered");
remind = true;
}
else if (digitalRead(MOTION_SENSOR) == LOW) {
digitalWrite(LED, LOW);
remind = false;
}
if (remind == false) {
// fills the screen with black to save power
// and also not draw attention when not needed
tft.fillScreen(ILI9341_BLACK);
while (remind == false) {
if (digitalRead(MOTION_SENSOR) == HIGH) {
digitalWrite(LED, HIGH);
//Serial.println("motion sensor triggered");
remind = true;
}
else if (digitalRead(MOTION_SENSOR) == LOW) {
digitalWrite(LED, LOW);
remind = false;
}
}
}
//only do things if the motion sensor gets tripped
if (remind == true) {
// first read the screen to see if he did anything
// Retrieve a point
TSPoint p = ts.getPoint();
//See if there's any touch data
if (p.z > MINPRESSURE && p.z < MAXPRESSURE) {
//Scale using the calibration #'s and rotate coordinate system
bx = map(p.y, TS_MINX, TS_MAXX, 0, tft.width());
by = map(p.x, TS_MINY, TS_MAXY, tft.height(),0);
if (hasEverything == false) {
// if he presses the button
if ((255 < bx && bx < 310) && (192 < by && by < 230)) {
// set hasEverything to true and execute that part of the code
hasEverything = true;
}
if (hasEverything == true) {
byeBtn();
delay(5000);
hasEverything = false;
yesBtn();
}
}
}
p = ts.getPoint();
//See if there's any touch data
if (p.z > MINPRESSURE && p.z < MAXPRESSURE) {
//Scale using the calibration #'s and rotate coordinate system
bx = map(p.y, TS_MINX, TS_MAXX, 0, tft.width());
by = map(p.x, TS_MINY, TS_MAXY, tft.height(),0);
Serial.print("bx = "); Serial.print(bx); Serial.print(" by = "); Serial.println(by);
// pressed add button
if (255 < bx && bx < 310 && 143 < by && by < 181) {
bx = 10;
by = 10;
adding = true;
addMode();
}
checkScroll();
}
// if he adds stuff from serial monitor
if (Serial.available() > 0) {
//// upload it onto the main display
add = Serial.readStringUntil('\n');
updated = updated + '\n' + ' ' + add;
Serial.println(updated);
yesBtn();
}
}
}
NOTE: After it is tested and confirmed to work with the Arduino, we will be adding in the IoT version of this code, which connects with Google API to return events without the need for input.
Schematic and Design Files
Schematic of the box at final demo
]]>
Joanne wanted to have control of her front yard lamp from inside her house. She also wanted her front steps to be better lit at night. We made a modular system that triggers her linear switch box for the lamp and also lights up LED tape for her stairs when she flips a switch inside her house. This system gives Joanne complete control of her front yard lights from a single switch inside. See here for documentation of our prototype and see here for documentation of our first meeting with Joanne.
What We Built
This overall photo shows all components of the project.
This video shows our project in action. When the user grips the photocell and blocks light from reaching it, the system is triggered and it turns on the LED strip and the servo motor rotates to move the toggle switch up. When our device is actually implemented at Joanne’s house the photocell won’t be squeezed by the user, rather the photocell will be focused on a lightbulb that is controlled by Joanne from inside her house.
The LED strip and servo motor assembly are the two main features of this project.
The electronic hardware that deal with voltage logistics and computation are shown in this photo.
This photo shows the 3D printed assembly that the servo motor operates. The part in grey is a model of Joanne’s linear toggle switch. The part in black is what we will implement at her house to drive the toggle switch.
Here is a closeup of the rack and pinion mechanism that is used to translate the servo’s rotational motion into linear. This is an important feature because we are using the servo’s rotational output to activate the toggle switch which requires linear motion.
The photocell is placed in a 3D printed housing to focus its view on the lightbulb at Joanne’s house that will be triggering the whole system.
Narrative Sketch
It is a cold winter night and Joanne is in her house when her brother comes home. She wants to turn on her front yard lights for him but one of the front yard lamps is controlled by a switch located outside. She isn’t dressed properly to go outside into the freezing weather conditions. Just as she begins to go put on her warm coat she realizes that two students from Carnegie Mellon just implemented a yard light system that allows her to control all of her front yard lights by a single switch inside. Bursting with delight, she smiles as she flips the switch that controls one of the lamps on her front yard. The photocell that the students place inside of this lamp changes resistance value due to the increased light from the lamp turning on. This triggers her newly implemented yard light system to do two things. One, rotate the servo motor which pushes the linear toggle switch up and turns on the second lamp on her front yard. Two, turn on the LED light strip and light up the previously unlit section of her front yard path. Her brother walks inside and then she flips her control switch again and a split second later all the lights on her front yard are back off.
How We Got Here
After our first prototype we knew we would have to order a lot of parts for the final device. Once essential part was the waterproof servo. Once this arrived the first thing we did was test it.
The new servo runs on a maximum of 6.8 volts and we used a power supply to test it.
It was a little different than the servo we were used to using because it had 270 degrees of rotation instead of 180. We knew that we would have to eventually adjust our code to control it but the more challenging problem was to change our mechanical design of our prototype’s 3D printed assembly to fit with the larger dimensions of the new servo. First, we took measurements of the servo and modeled it in SolidWorks.
We used a caliper to measure the servo and we only modeled the important features. We ignored all the little fillets and complex grooves.
Although it didn’t take that long to model the servo, it was still time wasted because we later found an exact CAD model on the servo manufacturer’s website. Having the CAD of our servo made it easier to design around. For our new design, we wanted to ensure that we were getting the maximum amount of torque output from our servo so that there would be no issue in terms of strength with moving the linear switch at Joanne’s house. To do this, we had to optimize the size of the gear that would attach to the servo. The smaller the gear, the higher the torque output is. Knowing that the maximum range of rotation for our servo is 270 degrees and that the range of motion for the linear switch is about 0.8 inches, we calculated how small the pitch diameter of our gear could be. We ended up with a pretty small gear that only had 8 teeth on it. It was only after we implemented this new gear into our CAD assembly that we realized the lower bound of our gear size was actually constrained by the width of the servo. Unless we made drastic design changes to our assembly, the gear diameter had to be slightly larger than the width of the servo or else it wouldn’t be able to mesh with the rack. So we designed a larger gear and then moved on to making design adjustments of other parts in our assembly. These adjustments included:
- Slightly reducing the dimensions on the rack for smoother gliding within its track
- Making the grip for the switch bigger and enclosed to maximize the contact area with the switch
- Raising the location of the servo relative to the rack so that it started at the top of the rack and allowed us to optimized the length of the rack.
- Moving the connection point for our rack and pinion assembly from the front face of the switch box cover to the side face.
- Reducing the weight of the rack and pinion assembly by cutting down unused wall space.
- Minor changes to increase structural integrity.
Final assembly of the rack and pinion mechanism that the servo drives.
We also made a significant design change to our switch box cover which is what our rack and pinion assembly will be mounted to. Our prototype cover enclosed the entire switch box. For the final design, we decided that covering the entire switch box was unnecessary and only increased the chances of our cover not fitting properly over the box. So we designed the new cover to be like a small hat for the switch box with a small wall coming down the side for the rack and pinion assembly to be bolted to. This was a minimalist solution to a problem we were very concerned about. The switch box had a lot of protruding bolts, inconvenient fillets, and other complexities induced by deterioration over time. We were worried that even with accurate measurements of the box, these features could throw off the fit of our cover and therefore, cause our switch grip to be misaligned. We think our simplified box cover greatly increases the probability of a proper fit but we still plan to bring some files and other tools with us when we implement our device at Joanne’s house in case additional adjustments need to be made.
Full assembly of Joanne’s linear toggle switch and the device that we designed to control this toggle switch. The part that is a darker grey shade plus the cylindrical component coming out of the bottom of it is a model of Joanne’s switch. The rest is what we will be implementing.
We used an Ultimaker to 3D print all of our parts.
Full assembly of our 3D printed parts. The part in grey is a model of Joanne’s switch and the parts in black constitute our solutions to controlling this switch with a servo.
To test the performance of our assembly we made a simple circuit to control the servo with a push button.
Our solution to control the linear toggle switch was now complete. We then moved onto the other main component of our project: controlling the LED light strip. We first hooked up our LED strip to a power supply to ensure that it was working.
Our LED strip runs on 12 volts. The lights are a cool white color.
We used an n-channel mosfet to turn on and off the LED strip. We were unsure about how to properly use a mosfet so we did a little research online and made a simple circuit on a breadboard and tried a few different configurations until it worked.
Test circuit using our mosfet.
With all of our hardware and electronic components properly working, we could then integrate everything together. To do this we first had to cut our electrical cords and wires to the proper lengths dictated by the measurements we had previously taken of Joanne’s front yard. To determine these lengths, it helped to visualize the layout of our entire system so we drew a rough sketch of it on a table.
Each component was methodically placed on the table to be in the relative position that they would be when we actually implement the entire system at Joanne’s house. This diagram helped us determine the four different cord lengths that we needed.
We cut the electrical cords to conservative lengths and then we were ready to solder everything together. We soldered a few connections but before long, it became hard to keep track of everything so we drew another diagram to visualize the entire circuit.
This diagram gave us a visual of almost all the connections that needed to be soldered.
We spent a whole night soldering but by the morning our circuit was complete and all components were working properly.
Completed circuit with all connections soldered.
Joanne and Rory at the final crit. Claire unfortunately could not be present because she got very sick the night before.
Conclusions and Lessons Learned
From the final crit we added two major changes and one minor one. The first being a potentiometer which told the code what number was the threshold between night and day. This was a simple addition until the Potentiometer was accidentally plugged into 12v and not the 5v. This broke the Arduino Nano and it had to be removed and a new one soldered in its place. The second major point was allowing Joanne the ability to choose what turning on her front porch light would trigger: the LEDs, the lever, both or none. The simplest method to complete this is having two switches, on representing the lever and one representing the LEDs. If both are switched off turning on the front porch light does nothing. If both are switched on turning on the front porch light triggers both. A challenge was incorporating this addition into the design that would maintain integrity in the weather. A decision was reached that the switches will mount through the top of the waterproof box. The final slight adjustment was making use of the other half of the breadboard by adding indicator lights, a set for daytime and nighttime, a set for the lever pushed and pulled and a set for when the LEDs were on and when they were off in order to help with install and troubleshooting.
From the final crit we received approval from many people. One person said that our device “really solved the problem” and that this could be adapted to solve a similar problem he is having. He was concerned that the “connections seem delicate and could malfunction”. This brings us to our discovery on how critical visual communication was (see the next paragraph). I believe the delicate parts he referred to were the small thin wires on the breadboard itself, something that would be protected by the waterproof box and since the waterproof box wasn’t present at the crit his concern makes sense. But maybe he was psychic and detected the problem that took Zack and Claire a good hour to figure out why the system suddenly electrically stopped working.
Many others shared the concern of weatherproofing and durability.
Some honorable mentions:
“This would be applicable in my home, but I don’t live in a house”
“Good luck installing it :)”
“A creative solution that I will really appreciate”
Something that was a major communication bump early in the process was the first showing of the prototype. Because we lost a group member and a part of the project along with her, we attached a button to the breadboard to show the motor being triggered. Without the photoresistor giving the visual information we were trying to communicate the idea didn’t get across easily. Instead the visual of the button trumped the verbal explanation. The big takeaway was to be as straightforward and simplistic in the explanations for the best possibly clarity. Next time we would have jumped immediately from the final crit to an analysis of the next step, purchasing the switches and getting it ready for installation. Since we waited till the weekend we were unable to install in time and pushed the date back to January. If we could do anything differently it would be Claire not needing to go to the E.R. the night before the project was due.
The overall takeaway is that there is always something you overlook and time needs to be allotted for the mistake. Successfully managing a project is not defined in the ability to predict mistakes but to accommodate for them.
Schematic
Code
/*
* Yard Lights
* Claire Mildred and Rory Hubbard
*
* This code activates and LED strip and a servo motor depending on a reading
* from a photocell. When the photocell reading is below a certain threshold
* the LED strip will be turned off and the servo will rotate to the 0
* position. When the photocell is higher than the threshold the LED strip will
* be turned on and the servo will rotate 80 degrees. No rights reserved.
*
*/
#include <Servo.h>
Servo myservo;
// establish pins
const int PHOTOCELL_READ = A0;
const int LED_CONTROL = 5;
// establish photocell threshold value to turn on or off the system
const int THRESHOLD = 800;
bool down; // this variable will keep track of the position of the servo
int photoVal; // initialize a new integer to store the photocell value
void setup() {
pinMode(PHOTOCELL_READ, INPUT);
pinMode(LED_CONTROL,OUTPUT);
myservo.attach(3,500,2500); // pulse width range of this servo is 500 - 2500
// microseconds
myservo.write(0); // start with servo down
down = true;
//Serial.begin(9600);
}
void loop() {
photoVal = analogRead(PHOTOCELL_READ); // acquire a reading from the
// photocell every loop
// Serial.println(photoVal);
if (photoVal >= THRESHOLD && down == true) {
myservo.write(80);
down = false; // servo has moved the toggle switch up
digitalWrite(LED_CONTROL,HIGH); // turn on LED strip
delay(100); // little delay to deter flickering and allow servo to reach
// final position
}
else if (photoVal < THRESHOLD && down == false) {
myservo.write(0);
down = true; // servo has moved the toggle switch down
digitalWrite(LED_CONTROL,LOW); // turn off LED strip
delay(100); // little delay to deter flickering and allow servo to reach
// final position
}
// no else statement because we only need to write to our servo and LED's
// one time when it changes state
}
]]>
For our final project, our team worked with Joseph, a semi-retired attorney and a Parkinson’s patient, in a collaborative effort to design and develop an assistive device that would be useful to him every day. Over the past several weeks, Team Joseph created a detachable cane assist that focused on mitigating some of Joseph’s freezing episodes, which are involuntary and temporary blocks of movement that can limit his everyday mobility. Typically, Joseph experiences freezing episodes when he is due for his next dose of medication. This accessory will be useful to him during this time when he may be more vulnerable to freezing.
For more information about our process, please read our prototyping documentation and our meeting documentation with Joseph.
What we built
This detachable cane accessory provides a visual cue (a red laser on the ground) and a haptic cue (vibrating disks). Both of these cues gives Joseph a specific target to focus on, which can help him move past a freezing episode. The cane accessory is a 3-D printed box that has buttons at the top to control the power (on/off switch), the visual cue (laser), and haptic cue (vibrating disks). In case Joseph needs to change the rechargeable batteries, we implemented a hinge and a latch that can open the box. As Joseph travels frequently to visit family across the country, we ensured that the batteries were TSA-safe.
We 3D printed a custom box that can attach to any of Joseph’s many canes. Joseph likes to buy new canes for different needs (e.g. a cane for snowy/slippery weather, a cane for walking on inclines, a cane for collapsible travel purposes, etc.)
Luckily, we were able to find a spare cane in the Phys Comp lab. This helped us decide early on that we wanted a detachable feature, rather than built in directly to one of Joseph’s canes.
We ordered spring clips online. However, one roadblock that we ran into was that the clips were slightly too large for the cane we had at hand. Since this was a standardized size (and the next size up/down would not be appropriate), we decided to implement a velcro band that can attach to the cane as well, and keep it from falling off forward, which was an incident that we had during one of our demos.
The very bottom contents of our box. This contains our soldered wirings to a protoboard to increase the box’s durability in the case that the cane is dropped. It also contains an Arduino Nano (power source not pictured).
The next layer of contents in our box. We used a scrap piece of thin 3D printer material to separate the backend wirings from the rest of the box. Pictured is the laser that is positioned at an angle to provide a visual cue that Joseph can follow and step over.
Top layer of contents in our box. We have a 3×2 battery pack powered by rechargeable dry-cell batteries that are TSA-approved. To keep the battery from jiggling around in the box, we cut a piece of foam to secure all of the box’s contents in place. We placed the battery at the top of the box for ease of reach once Joseph needs to change the batteries.
A close-up view of the laser that we embedded into the box. We used a hot glue gun to secure its position in case of a fall. On the inside right part of the box, we also glued 5 vibrating disks that will vibrate the box and the handle to provide Joseph a haptic cue. We tested the vibrations on Joseph to ensure that they were strong enough.
The top of the box contains three input buttons and switches. The switch turns the assistive device on/off; the off option allows us to save battery when Joseph does not need the assistance. The red button turns on the laser, and the black button turns on the vibrations. We made these cues independent of another per Joseph’s suggestion.
The above video shows how each of the buttons work. We designed the buttons to sit on top of the box so that Joseph could reach them with minimal movement (we confirmed with him that while he was in a freezing episode, he was able to move his fingers to reach these buttons).
Despite being a Parkinson’s patient, Joseph is still very active and travels frequently to visit his family in San Francisco. When Joseph goes to the airport, he brings a different cane with him — a collapsible one that will fit in his carry-on bag. Joseph also brings his detachable Cane Assist that can attach onto any one of his canes flexibly. He goes through TSA successfully.
In the latter half of the day, Joseph’s symptoms tend to worsen. Before his dopamine medication is due, he is more prone to freezing episodes. When he is navigating the airport, Joseph experiences a freezing episode, where he struggles to coordinate where to move next. He turns on the Cane Assist and pushes the laser button, as well as the vibrator button, which provides him both visual and haptic cues to stimulate his brain and help direct his next steps. With the Cane Assist’s help, Joseph is able to move past the episode. He remembers to take his medication and successfully reaches his destination.
How we got here
Along the journey of building Joseph’s Cane Assist, we made mistakes, discoveries, and pivots.
One big challenge that our team faced was figuring out how to build a custom box for Joseph. Since none of us had experience with 3D fabrication, we had to obtain outside help (thank you to one of Catherine’s friends).
parts measurement
box design details
We faced several 3D printing mishaps, like the printer jamming overnight, so in the meantime we built an acrylic backup prototype.
As a backup, we laser cut an acrylic box to fit all of our hardware. We wanted it to be a closed-box solution.
However, after laser cutting the pieces and gluing them together, we realized that the acrylic box (although visually appealing) was extremely heavy and would add significant weight to the cane. Consequently we reinvested our efforts into 3D printing our box. Luckily, one of our classmates, Roly, offered to print our box for us with his at-home 3D printer (shoutout to Roly — thank you!).
Our 3D printed box (at last) with Roly’s printing help. This material was much lighter and didn’t add noticeable weight to the cane.
The 3D printed box that Roly helped us print out was lightweight and much better suited to our needs, especially considering that we needed to add in a battery pack (significant weight) and other hardware components. However, finding a suitable hardware solution was not the last of our worries.
Another significant challenge we faced was finding a suitable battery pack. Because of our electrical components (laser, vibrators) we needed at least 6 batteries, or 7.2 volts. In the classroom, we were able to find some spare battery holders, which were either 6 in a line, or in a 3×2 configuration. Some of these holders were faulty and/or had physical gaps in the electrical connection.
This was our 3×2 battery pack that had a missing electrical connection. Zach kindly helped us solder our wire directly onto an electrical connection to get it working. However, during a demo, our box was dropped and knocked some things astray in our battery holder.
Zach helped us solder a wire to correct some strange electrical disconnections in this battery holder. During our first client presentation, our Cane Assist was dropped and we determined that the battery pack was an issue again. We had to last-minute re-order a new battery pack in this 3×2 configuration (due to space constraints in our box). We also worked on ensuring that our box would not fall during usage again.
To remedy this problem, our team found some spare foam and cut a hole in the middle to fit out battery pack. This foam would act as a physical cushion that would hopefully absorb impact in case the cane assist was dropped. The foam would also prevent the battery from making unnecessarily loud vibrating sounds when the vibrator button was turned on.
We added foam stuffing to prevent the battery from moving around inside the box and making excessive noise when the vibrating button was turned on.
We also realized that the cane assist had fallen during the demo because the spring clamps that we ordered did not perfectly fit the cane that we had at hand. Since these sizes were standardized, we were unable to find a size that was a closer fit to the cane diameter. Instead, we installed a velcro strap at the back of the box that would act as a third flexible clamp to hold the box securely to the cane.
We installed a velcro strap to the back of the box to more securely fasten it to any cane.
We thought adding a velcro strap would be the best solution, because cane diameters cane vary, and it would not be possible to find a single clamp that fits all the different sizes of Joseph’s many different canes.
Since this cane assist is a detachable solution that would be carried around, we wanted our batteries to be rechargeable. We purchased dry-cell rechargeable batteries on Amazon that were TSA-friendly. However, another challenge was designing the 3D-printed box such that it could be opened and closed, since Joseph would need to replace the batteries from time to time.
Per Zach’s suggestion, we devised a hinge-and-latch solution, where the front wall of the box would have a hinge that could allow the wall the swing open. There would be an additional latch at the outside of the box’s front that would keep the “door” shut.
Our sketch of the hinge-and-latch solution to make our box accessible, so that Joseph can replace the batteries when needed.
We had some extra help on this part. 
(Thanks Zach)
The finished hinge-and-latch solution.
We decided to purchase a pinless plastic hinge to reduce mechanical complexity. Combined with the latch, installed at the top of the door and the box’s side, the hinge-and-latch mechanism worked fluidly and easily, without popping open and keeping all the contents of the box in place.
Lastly, another challenge that we had was getting the laser to position correctly inside our box. If the laser was installing pointing straight down, it would not give sufficient distance from the cane to be a walking cue that Joseph could use to “step” over.
Using some complex measuring tools (a protractor), we tried to measure at what angle the laser should be pointing to give sufficient distance from the cane and be a walking target.
Despite our best measuring efforts, it was difficult to build out a model in our 3D modeling software that would allow a sufficient angle. Previously, we had a separate part that would help anchor the angling component, but this ended up being too large and taking up valuable space inside our cane assist box. As an alternative solution, we filed the hole for the laser to be slightly larger, and used a hot glue gun to properly anchor the laser at the right angle.
Conclusion and lessons learned
For most of us, this was our first client project. From this project, we learned that the initial client meeting is an incredibly important meeting. Our first meeting with Joseph lasted well over two hours, and while we didn’t stick to a script, perhaps in the future we would better outline the structure of client meetings, so that it’s clear for both sides what information needs to be exchanged. Joseph came into the meeting very prepared, with printouts of design ideas he already had and suggestions/solutions for cane accessories that we could build. During the meeting, it felt like we had a lot of ideas that we could potentially build; however, afterwards, we realized that many of these were mechanical solutions, and that it would be difficult to incorporate an electrical component. Therefore, for initial client meetings, we learned that it’s important to direct the conversation to learn about the client’s problems rather than solutions right away; this means understanding his background, habits, and pain points. Honing specifically in on understanding these points would have better helped expand the breadth of potential design solutions we could have worked on.
Prior to the initial meeting, it also would have been helpful for us to have thoroughly researched the symptoms of Parkinson’s disease beforehand. We had been mostly focused on gathering interview questions for Joseph, but hadn’t known what kinds of problems that the client had (perhaps it’s a good idea to give a brief background of each client prior to the initial meeting for future Phys Comp students). With some research beforehand, we believe we could have also helped expand the breadth of design ideas by asking Joseph how he experiences some specific symptoms associated with Parkinson’s.
During our final crit, we received mostly positive feedback. Some people mentioned that this idea is one that could potentially be monetized or purchased by a medical device company, which was encouraging to hear. We also received some concerns about weatherproofing — while one of our initial goals was to keep the box weather proof, we realized that it would not be possible after we devised the latch-and-hinge mechanism (the box would not be watertight with a door). For future iterations of the project, we would look into designing a custom box that would be resistant to Pittsburgh’s random weather. Additionally, we would add more custom functionality to the pacing of the vibrations, perhaps with a potentiometer, such that Joseph can tune the rhythm to his liking.
Overall, we really enjoyed working on this project, and we’re excited to hand-off the project to Joseph so he is able to use it as an everyday assistive device.
Technical details
/*
* Project Title: Cool Cane
* Name: Catherine Yu, Linda Xia, Joey Santillo
* Description: There are two buttons. One controls on/off the haptic cue(vibrator) and
* the other controls on/off of the visual cue(laser).
*/
//PIN ASSIGNMENTS
const int LASERPIN = 5;
const int VIBRATORPIN1 = 8;
const int VIBRATORPIN2 = 9;
const int VIBRATORPIN3 = 10;
const int VIBRATORPIN4 = 11;
const int VIBRATORBUTTON = 4;
const int LASERBUTTON = 3;
//CUSTOMIZABLE VARIABLES
const int vibrateInterval = 1000;
const int debounceDelay = 10;
//VARIABLES NEEDED FOR LATER
bool LASERNeeded = false;
unsigned long lastLASERTime = 0;
int lastLASERState = HIGH;
bool VIBRATENeeded = false;
bool vibrating = false;
unsigned long lastCUETime = 0;
int lastCUEState = HIGH;
unsigned long currTime = 0;
void changeVibrate(){
if (currTime-lastLASERTime>=vibrateInterval){
if (vibrating){
digitalWrite(VIBRATORPIN1, LOW);
digitalWrite(VIBRATORPIN2, LOW);
digitalWrite(VIBRATORPIN3, LOW);
digitalWrite(VIBRATORPIN4, LOW);
vibrating = false;
}
else{
digitalWrite(VIBRATORPIN1, HIGH);
digitalWrite(VIBRATORPIN2, HIGH);
digitalWrite(VIBRATORPIN3, HIGH);
digitalWrite(VIBRATORPIN4, HIGH);
vibrating = true;
}
lastLASERTime = currTime;
}
else{
if(vibrating){
digitalWrite(VIBRATORPIN1, HIGH);
digitalWrite(VIBRATORPIN2, HIGH);
digitalWrite(VIBRATORPIN3, HIGH);
digitalWrite(VIBRATORPIN4, HIGH);
}
else{
digitalWrite(VIBRATORPIN1, LOW);
digitalWrite(VIBRATORPIN2, LOW);
digitalWrite(VIBRATORPIN3, LOW);
digitalWrite(VIBRATORPIN4, LOW);
}
}
}
bool checkDebounce(String indicator, int state){
//returns true if needs an update
//false otherwise
if (indicator == "LASER"){
if (currTime-lastLASERTime>=debounceDelay && state != lastLASERState){
lastLASERState = state;
lastLASERTime = currTime;
return true;
}
return false;
}
else{
if (currTime-lastCUETime>=debounceDelay && state != lastCUEState){
lastCUEState = state;
lastCUETime = currTime;
return true;
}
return false;
}
}
void setup() {
Serial.begin(9600);
pinMode(LASERPIN,OUTPUT);
pinMode(VIBRATORPIN1,OUTPUT);
pinMode(VIBRATORPIN2,OUTPUT);
pinMode(VIBRATORPIN3,OUTPUT);
pinMode(VIBRATORPIN4,OUTPUT);
pinMode(VIBRATORBUTTON,INPUT_PULLUP);
pinMode(LASERBUTTON,INPUT_PULLUP);
}
void loop() {
currTime = millis();
if(LASERNeeded){
digitalWrite(LASERPIN, HIGH);
}
else{
digitalWrite(LASERPIN, LOW);
}
if(VIBRATENeeded){
changeVibrate();
}
else{
digitalWrite(VIBRATORPIN1, LOW);
digitalWrite(VIBRATORPIN2, LOW);
digitalWrite(VIBRATORPIN3, LOW);
digitalWrite(VIBRATORPIN4, LOW);
}
if (checkDebounce("LASER", digitalRead(LASERBUTTON)) && digitalRead(LASERBUTTON)==LOW ){
if (!LASERNeeded){
LASERNeeded = true;
}
else{
LASERNeeded = false;
}
}
if (checkDebounce("VIBTRATOR",digitalRead(VIBRATORBUTTON)) && digitalRead(VIBRATORBUTTON)==LOW){
if (!VIBRATENeeded){
VIBRATENeeded = true;
}
else{
VIBRATENeeded = false;
}
}
}
Schematic
You can view the Google drive link to our 3D models (STL files) here.
Thank you, Joseph!
Our team had a wonderful time working with Joseph on this cane assist.
This project is a Bingo-style game for Jeffrey to play with his 6-year-old granddaughter, Stella. This game, partially disguised as trivia, would be used as a tool to spark conversations about interests and personal topics as a way to get both young kids and adult family members to know each other better. See here for documentation of our initial meeting with Jeffrey and see here for documentation of our prototype.
What we built
We built an electronic board-game, with very simple rules, which could be used for both a 73 year old and a 6 year old. The game-play is a mix between Trivial Pursuit, Bingo and Truth or Dare.
Before the game starts, each Player writes down a set of questions for the other player to answer. The use of this analog method is to create a constantly evolving game, which grows and complexifies as Stella does. Player 1 asks a question from their stack of cards, and player 2 picks a spot on the board and attempts to answer. If answered correctly, player one presses their green button which locks the location on the button pad (as seen in the image below). If the answer is wrong, however, the illuminated button disappears, and player 2 must try again in the next round. This format continue until one player obtains 5 lit-up buttons in a row.
For a long game play, penalty cards such as “Tell me a story in song” or “Invent a short dance routine”, may be given to the person who has answered wrong at the end of each turn. For a short game play, the penalty card may be given at the end of the game.
Player 1 reading a question card, while player 2 chooses a position on the button pad.
Detail shot of the LCD screen, giving the players indications: “Green asks, Pink answers.” You also get a sense of the box’s transparency, which was a customer request made by Jeffrey.
Detail shot of the button pad, correct and incorrect buttons, as well as the LCD screen.
Detail shot of the speakers and box design on the back face.
Narrative Sketch
Stella gets dropped off by her single father at Jeffrey’s house, for their weekly Wednesday hangout. Stella would like to play a game, but Jeffrey isn’t sure how to successfully entertain a 6-year girl. Instead of having his wife help him out, he pulls out their customized board-game.
Stella likes to start, because she enjoys the pretty colors the button pad generates. As the game starts, a rainbow pattern illuminates the board, making Stella smile.
The first step of the game is for each player to write down a list of personalized questions on their assigned cards, which can be re-used or re-written as pleased. The aim of these questions is to generate conversation, and learn something from one another. Then, Stella starts by pressing a button of her chosen location on the button pad. Jeffrey asks her a question from his green deck of cards .
Player 1 chooses a location on the board.
Stella attempts to answer the question. If answered correctly, Jeffrey will select the green button, locking in the position on the buttonpad. If the answer is incorrect, Jeffrey presses the red button, and the button’s light goes out. Stella can then try again during the next round. In a long-play game, Stella would then have to perform a penalty found on the white stack of cards.
Player 2 reads the card from their stack, and prompts Player 1 to answer. If correct, Player 2 presses the “correct” button, and the button pad stays lit.
After Stella’s turn, Jeffrey can pick a position and attempt to answer a question. The game goes on until a player is able to line up 5 of their colored buttons on the pad. During a short play game, the loser must perform a penalty found on the white stack of cards, only at the end of the game.
When the game is over, the orange “reset” button can be pressed, a rainbow pattern appears and the board is cleared.
How we got here
Initial Meeting and Brainstorm
On November 1st, 2018, our group – Chloé Desaulles, Jiatian Sun, Jianxiao Ge – firstly met up with Jeffrey, who is an older man participating in the CMU Osher program, with the intention of building a useful device for him.
We had a meeting and interviewed Jeffrey in the university cafe. After the first interview, we started by brainstorming ideas to make Jeffrey’s life easier or more pleasurable. He did not seem to need anything fixed, so we tried to look into ways of enhancing good things in his life. It became clear very quickly that he kept gravitating towards his granddaughter Stella, and the time he and his wife get to spend with her. We additionally noticed that Jeffrey might not perfectly know how to interact with a six-year-old, often letting his wife entertain her.
Our main challenge was creating an interesting and fun game for such a wide age gap. Making a game intuitive and worth playing was a hard thinking exercise considering none of us had any game design experience. In addition, we needed to add physical computing elements into the game, which made it even harder.
Finally, we were inspired by the traditional game Gomoku & Bingo and decided to create a similar game with trivia elements, which match Jeffrey’s interests, and a simple and colorful button interface, with a changeable gameplay so the game can be adaptable as Stella grows up.
Preliminary sketches of our game board.
Prototype and the Second Meeting
After settling on the concept and gameplay, we started working on the layout and board design, option for big buttons with fun interactions, which would make the game more interesting to a 6-year-old. However, because large buttons took up too much space, we ended up with a medium sized button.
Working on the conceptualization and physical design/implementation of the game board.
We then started programming the colored buttons, LCD screen, and button pads separately and finally got together to merge our individual codes.
Jianxiao and Jiatian working on merging code sections.
The LCD screen welcoming Jeffrey and Stella to their custom game.
One of the biggest challenges we met during prototype was the core part of our game, the button pad. The RGB keypad we needed was out of stock at that time. We contacted the manufacturer Adafruit and received a reply that it would be two weeks before we could have the keypad available. Therefore, we had to order the LED version of the pad as an alternative in order to finish the prototype on time.
Setup of our first button pad trial with LED keypad.
Button Pad is able to light up according to the color of LED beneath it.
The completed first prototype.
We met Jeffrey again after the prototype presentation. Jeffrey seemed very excited by our prototype and believed he and Stella would have fun playing the game. He was also happy about our choices in tactility and color and even suggested we add sound (which is something we had been shying away from because we thought it might be irritating Jeffrey and his wife). What’s more, he gave us a lot of input when it came to the aesthetics of the board, suggesting we should make it out of clear acrylic so that Stella could see the colorful wiring on the inside. Some other fabrication advice he and Zach gave us included building a box to hold question and penalty cards. The crit was useful for our team and allowed us to move confidently forward with our design.
Jeffrey giving us feedback on our prototype.
Final Fabrication
After the second meeting, we adjusted the game based on the feedback. For example, combine the card box with the game board, add the game sound effect, and remove people’s name from the instruction on the LCD screen to involve other family members as well.
Chloé and Jiatian adjusting the game after the second meeting.
Another good news was that RGB Keypad, which was out of stock before, had been replenished. With the help of Zach, we got the keypad in time and immediately started to adjust the code as well as the soldering and testing.
Soldering four 4*4 RGB Keypads together.
At the same time, we started the design of the game box and material preparation. According to Jeffrey’s suggestion, we chose the white fog-faced acrylic as the main material to highlight the colorful buttons and create a light and transparent visual effect.
We sketched the game board by hand and determined the general layout. After actually measuring the size of each part, we drew it exactly one-to-one and then redrew it in our laptop using Autocad and Adobe Illustrator.
Rough hand sketch of game board design and dimensions.
Drawing it one-to-one after measuring the size of each part.
While Jiatian was in charge of the code adjustment and part testing, Chloé and Jianxiao took the acrylic to the laser cut studio. The laser cutting process was not as smooth as expected. Because of the measuring error, the buttons and the LCD screen couldn’t be embedded smoothly. So we measured again, adjusted the size and then carried out a second laser cut, finally got the panel in line with the requirements and started the assembly.
During the assembly, we found that the card box was smaller than expected, so the card could only be erected. But we thought it would make it easier to pull out the cards and make the game go more smoothly, so we didn’t redo the box.
On December 4th, a month after the initial meeting, we had the final presentation and met Jeffrey again. We were delighted to see that he was very pleased with our final product. He played the game with us and was ready to give it to his granddaughter Stella as a Christmas present.
Jeffrey playing the game with us during the final desk crits.
Here is our team! (From left to right: Jiatian Sun, Jeffrey, Chloé Desaulles, Jianxiao Ge)
Conclusions and lessons learned
To our surprise, almost all of the oral feedbacks we received during the final crit were positive, whether it was about the experience of the game or the design of the box itself. (which, of course, could be because people were too shy to make suggestions in person). We were happy to see that older people seemed to be very interested in the game. Some of them even thought that it had commercial value. Since the question cards can be written by the players themselves, it gives the game more possibilities. It can be a bridge for family members to communicate and learn from each other, even if there is a huge age gap.
After the crit, we also see a lot of very good advice from the written feedback. For example, “It would be useful to add the name/instructions onto the game.” Because of the novel mechanics of the game, people may be confused when they start to play. Although we try to give each step a hint on the LCD screen, some instructions are not clear and accurate due to the size of the screen. At the same time, an extra game description may help people understand the game faster.
Here is another one: “It could be smaller/thinner – effects portability. Would be nice if users could choose which colors they are in the button pad – maybe they have a favorite color.” The final volume of the game box was indeed larger than we expected. In order to prevent the electronic components from being unable to fit, we reserved much space during the box design. But when we actually assembled it, we found that most of the space in the box was empty. If we can make full use of space, the game can be greatly improved in portability (although the portability may be not that important for a family game box). For the color part, we did take Jeffrey and Stella’s preferences into account and set up keyboard lighting based on their favorite colors. It might be a better idea to let players themselves choose the own lighting colors.
We didn’t have much experience working with older people before, but the process of working with Jeffrey was very pleasant. He was very talkative and provided us with many practical suggestions, which helped us adjust in different stages and finally completed a work that satisfied us all. Due to the time limit, our initial interview could only be held in the university cafe. Although we talked a lot, it seemed difficult for us to get Jeffrey’s demand quickly from his conversation, which made us struggle about where to start in the brainstorm. Next time we have the opportunity to cooperate with the elderly, maybe we will choose their home as the interview place. On the one hand, it can make them feel more comfortable and be willing to talk more, on the other hand, it is more convenient for us to observe the details of their life so that we can understand their needs more quickly.
All in all, this project is an interesting interdisciplinary cooperation experience. We had a lot of brainstorming in the early stage and repeatedly discussed and improved the design scheme according to Jeffrey and Zach’s feedback. In the post-fabrication stage, we divided the work according to everyone’s background and expertise, and finally achieved satisfactory results. If you have any more questions or suggestions, please feel free to contact us by email: jianxiag@andrew.cmu.edu, thanks!
Technical details
Schematic
Code
/*
* This code makes reference to sample code of Adafruit seesaw liborary
* whose source code can be viewed at https://github.com/adafruit/Adafruit_Seesaw
*
* Game Logic part Author: Caroline Sun & Jianxiao Ge
* This is a game is similar to Bingo. To create a enjoyable gaming experience, we add extra
* features like playing sound sound to the game.
*/
#include "Adafruit_NeoTrellis.h"
#include <LiquidCrystal_I2C.h>
LiquidCrystal_I2C lcd(0x27, 16, 2);
#define DEBUG 1
// TONES ==========================================
// Start by defining the relationship between
// note, period, & frequency.
#define c 3830 // 261 Hz
#define d 3400 // 294 Hz
#define e 3038 // 329 Hz
#define f 2864 // 349 Hz
#define g 2550 // 392 Hz
#define a 2272 // 440 Hz
#define b 2028 // 493 Hz
#define C 1912 // 523 Hz
// Define a special note, 'R', to represent a rest
#define R 0
#define Y_DIM 8 //number of rows of key
#define X_DIM 8 //number of columns of keys
//Define game states
#define QUESTION 0
#define ANSWER 1
#define NEXT 2
//Define Answer to Questions
#define YESJ 2
#define YESS 4
#define NOJ 3
#define NOS 5
//Define button pins
#define RESET 6
#define NEXTJ 5
#define NEXTS 6
//Define player states
#define STELLA 0
#define JEFFREY 1
//Define Board Value
#define MIDDLE 3
#define INVALID (-1)
#define BINGOSIZE 5
//Define speaker pins
#define SPEAKER 9
//Color value
int val = 125;
//Declare game States
int isStart;
int state;
int player;
int currSquare;
int scoreS;
int scoreJ;
int SIZE = 8 ;
int totalSize = SIZE * SIZE;
int board[64];
// Set overall Speaker configuration
long tempo = 10000;
int pause = 1000;
int rest_count = 100;
// MELODY and TIMING =======================================
// melody[] is an array of notes, accompanied by beats[],
// which sets each note's relative length (higher #, longer note)
//This is the tune of correct answer
int successMelody[] = { c, e, g, C, C, C};
int successBeats[] = { 8, 8, 8, 8, 8,8};
int SUCCESS_MAX_COUNT = sizeof(successMelody) / 2;
//This is the tune of wrong answer
int failureMelody[] = { g, e, d, c, c};
int failureBeats[] = { 8, 8, 8, 8, 8};
int FAILURE_MAX_COUNT = sizeof(failureMelody) / 2;
//This is the tune of winning
int winMelody[] = { e, e, e, e, f, f, f, f, g,g, g, g, g, g, g, g, f, f, f, f, e,e, e, e, d, d, d, d};
int winBeats[] = { 16, 16, 16, 16, 16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16};
int WIN_MAX_COUNT = sizeof(winMelody) / 2;
//This is the tune of reset
int resetMelody[] = { c, c, c, c, g, g, g, g, c, c, c, c, g, g, g, g};
int resetBeats[] = { 16, 16, 16, 16, 16,16,16,16, 16, 16, 16, 16,16,16,16,16};
int RESET_MAX_COUNT = sizeof(resetMelody) / 2;
// PLAY TONE FUNCTION ==============================================
// Pulse the speaker to play a tone for a particular duration
void playTone(int tone_, int beat, int duration) {
long elapsed_time = 0;
if (tone_ > 0) { // if this isn't a Rest beat, while the tone has
// played less long than 'duration', pulse speaker HIGH and LOW
while (elapsed_time < duration) {
digitalWrite(SPEAKER,HIGH);
delayMicroseconds(tone_ / 2);
// DOWN
digitalWrite(SPEAKER, LOW);
delayMicroseconds(tone_ / 2);
// Keep track of how long we pulsed
elapsed_time += (tone_);
}
}
else { // Rest beat; loop times delay
for (int j = 0; j < rest_count; j++) { // See NOTE on rest_count
delayMicroseconds(duration);
}
}
}
//Play Melody function
void playMelody(int* pitches, int* beats, int count){
// Set up a counter to pull from melody[] and beats[]
int tone_,beat,duration;
for (int i=0; i<count; i++) {
tone_ = pitches[i];
beat = beats[i];
duration = beat * tempo; // Set up timing
playTone(tone_,beat,duration);
// A pause between notes...
delayMicroseconds(pause);
}
}
//create a matrix of trellis panels
Adafruit_NeoTrellis t_array[Y_DIM/4][X_DIM/4] = {
{ Adafruit_NeoTrellis(0x2E), Adafruit_NeoTrellis(0x2F) },
{ Adafruit_NeoTrellis(0x30), Adafruit_NeoTrellis(0x31) }
};
//pass this matrix to the multitrellis object
Adafruit_MultiTrellis trellis((Adafruit_NeoTrellis *)t_array, Y_DIM/4, X_DIM/4);
//Print Board Function for testing
void printBoard(){
Serial.println("board");
for(int i = 0;i < totalSize;i++){
Serial.print(board[i]);
Serial.print(" ");
}
Serial.println("");
}
//Helper function that cleans up a array
void cleanArr(int *arr, int n){
for(int i = 0; i < n;i++){
arr[i] = 0;
}
}
//Write a two-line message to the LCD screen
void writemessage(String message1, String message2){
lcd.backlight();
lcd.setCursor(0, 0);
lcd.print(message1);
lcd.setCursor(0, 1);
lcd.print(message2);
}
//Turn on the button first being pressed
int turnOnFirstPressed(int num){
int val = 125;
if(player == JEFFREY) val = 255;
trellis.setPixelColor(num, Wheel(val));
trellis.show();
return num;
}
// Input a value 0 to 255 to get a color value.
// The colors are a transition r - g - b - back to r.
uint32_t Wheel(byte WheelPos) {
if(WheelPos < 85) {
return seesaw_NeoPixel::Color(WheelPos * 3, 255 - WheelPos * 3, 0);
} else if(WheelPos < 170) {
WheelPos -= 85;
return seesaw_NeoPixel::Color(255 - WheelPos * 3, 0, WheelPos * 3);
} else {
WheelPos -= 170;
return seesaw_NeoPixel::Color(0, WheelPos * 3, 255 - WheelPos * 3);
}
return 0;
}
//Check if one index is in bound of a board
bool inBound(int x, int y, int width, int height){
return x>=0 && y >=0 && x < width && y <height;
}
//define a callback for key presses
TrellisCallback blink(keyEvent evt){
if(evt.bit.EDGE == SEESAW_KEYPAD_EDGE_RISING){
//Wheel(map(evt.bit.NUM, 0, X_DIM*Y_DIM, 0, 255))
Serial.print("blink! ");
Serial.print(evt.bit.NUM);
Serial.print(" ");
Serial.print(val);
Serial.println(" prit");
trellis.setPixelColor(evt.bit.NUM, Wheel(val));
val = val>125? 125: 225;
}//on rising
else if(evt.bit.EDGE == SEESAW_KEYPAD_EDGE_FALLING)
trellis.setPixelColor(evt.bit.NUM, 0); //off falling
trellis.show();
return 0;
}
//The function that checks if there exists winner on the board
bool checkWinner(){
int rowCorrect[2];
int colCorrect[2];
int diagDownCorrect[2];
int diagUpCorrect[2];
int currCont[2] = {0,0};
cleanArr(rowCorrect,2);
cleanArr(colCorrect,2);
cleanArr(diagUpCorrect,2);
cleanArr(diagDownCorrect,2);
cleanArr(currCont, 2);
//Check if there is bingo in a row
for(int i = 0; i < SIZE; i++){
for(int j = 1; j < SIZE; j++){
int currP = board[i * SIZE + j];
if(board[i * SIZE + j] == board[i * SIZE + j -1] && board[i * SIZE + j]!=INVALID){
currCont[currP] += 1;
if(currCont[currP] > rowCorrect[currP]) rowCorrect[currP] = currCont[currP];
}
else{
cleanArr(currCont, 2);
}
}
}
cleanArr(currCont, 2);
//Check if there is Bingo in a column
for(int j = 0; j < SIZE; j++){
for(int i = 1; i < SIZE; i++){
int currP = board[i * SIZE + j];
if(board[i * SIZE + j] == board[(i-1) * SIZE + j] && board[i * SIZE + j]!=INVALID ){
currCont[currP] += 1;
if(currCont[currP] > colCorrect[currP]) colCorrect[currP] = currCont[currP];
}
else{
cleanArr(currCont, 2);
}
}
}
//Check if there is bingo in a diagonal
for(int xShift = -SIZE +1; xShift<SIZE;xShift++){
cleanArr(currCont, 2);
for(int diagX = 1; diagX<SIZE;diagX++){
int xPos = diagX + xShift;
int yPos = diagX;
int prevX = xPos -1;
int prevY = yPos -1;
if(!inBound(xPos,yPos,SIZE,SIZE) || !inBound(prevX,prevY,SIZE,SIZE)){
cleanArr(currCont,2);
continue;
}
int currP = board[yPos * SIZE + xPos];
if(currP == board[prevY * SIZE + prevX] && currP!=INVALID){
currCont[currP] += 1;
if(currCont[currP] > diagDownCorrect[currP]) diagDownCorrect[currP] = currCont[currP];
}
else{
cleanArr(currCont, 2);
}
}
}
for(int xShift = 0; xShift<2 * SIZE -1;xShift++){
cleanArr(currCont, 2);
for(int diagX = 1; diagX<SIZE;diagX++){
int xPos = -diagX + xShift;
int yPos = diagX;
int prevX = xPos +1;
int prevY = yPos -1;
if(!inBound(xPos,yPos,SIZE,SIZE) || !inBound(prevX,prevY,SIZE,SIZE)){
cleanArr(currCont,2);
continue;
}
int currP = board[yPos * SIZE + xPos];
if(currP == board[prevY* SIZE + prevX] && currP!=INVALID){
currCont[currP] += 1;
if(currCont[currP] > diagUpCorrect[currP]) diagUpCorrect[currP] = currCont[currP];
}
else{
cleanArr(currCont, 2);
}
}
}
int sScore = max(colCorrect[STELLA],max(rowCorrect[STELLA],max(diagDownCorrect[STELLA],diagUpCorrect[STELLA])));
int jScore = max(colCorrect[JEFFREY],max(rowCorrect[JEFFREY],max(diagDownCorrect[JEFFREY],diagUpCorrect[JEFFREY])));
//Display the winner of the game
if(sScore >= BINGOSIZE-1 && jScore >=BINGOSIZE-1){
Serial.println("Both of you wins!");
writemessage(" Both of you "," Wins! ");
playMelody(winMelody,winBeats,WIN_MAX_COUNT);
return true;
}
if(sScore >= BINGOSIZE-1){
Serial.println("Pink wins!");
writemessage(" Pink "," Wins! ");
playMelody(winMelody,winBeats,WIN_MAX_COUNT);
return true;
}
if(jScore >= BINGOSIZE-1){
Serial.println("Green wins!");
writemessage(" Green "," Wins! ");
playMelody(winMelody,winBeats,WIN_MAX_COUNT);
return true;
}
return false;
}
//The callBack function on the Trellis Buttons
//This function tells the Button to update the game state,
//if it is pressed when someone is answering question
TrellisCallback gameControl(keyEvent evt){
if(isStart){
// go through every button
if(state == QUESTION){
if(evt.bit.EDGE == SEESAW_KEYPAD_EDGE_RISING){
currSquare = turnOnFirstPressed(evt.bit.NUM);
if(currSquare>=0){
state = ANSWER;
}
}
}
}
}
//Setup the game
void setup() {
Serial.begin(9600);
//while(!Serial);
if(!trellis.begin()){
Serial.println("failed to begin trellis");
while(1);
}
//Register all buttons
pinMode(YESJ,INPUT_PULLUP);
pinMode(YESS,INPUT_PULLUP);
pinMode(NOJ,INPUT_PULLUP);
pinMode(NOS,INPUT_PULLUP);
pinMode(NEXTJ,INPUT_PULLUP);
pinMode(NEXTS,INPUT_PULLUP);
pinMode(RESET,INPUT_PULLUP);
//Register speaker
pinMode(SPEAKER,OUTPUT);
//Initialize LCD
lcd.init(); // initialize the lcd
// Print a message to the LCD.
lcd.backlight();
lcd.setCursor(0, 0);
lcd.print("Pink & Green");
lcd.setCursor(0, 1);
lcd.print("Welcome to BINGO");
//Clear board
for(int i = 0; i< SIZE * SIZE;i++){
board[i] = -1;
}
//Go through each button on the board and display different color
for(int i=0; i<Y_DIM*X_DIM; i++){
trellis.setPixelColor(i, Wheel(map(i, 0, X_DIM*Y_DIM, 0, 255)));
trellis.show();
delay(20);
}
//Setup the trellis button pad
for(int y=0; y<Y_DIM; y++){
for(int x=0; x<X_DIM; x++){
//activate rising and falling edges on all keys
trellis.activateKey(x, y, SEESAW_KEYPAD_EDGE_RISING, true);
trellis.activateKey(x, y, SEESAW_KEYPAD_EDGE_FALLING, true);
trellis.registerCallback(x, y,gameControl);
trellis.setPixelColor(x, y, 0x000000); //addressed with x,y
trellis.show(); //show all LEDs
delay(20);
}
}
//Randomize a player to start the game
float r = random()%2;
if( r > 0){
player = STELLA;
}
else{
player = JEFFREY;
}
//Reset Game State
isStart = 1;
state = QUESTION;
scoreS = 0;
scoreJ = 0;
currSquare = -1;
//Display welcome message
writemessage("Pink & Green","Welcome to BINGO");
}
void loop() {
trellis.read();
delay(20);
//If Rest Button is pressed, reset the game
if(digitalRead(RESET)<1){
writemessage(" Reset! "," ");
delay(500);
playMelody(resetMelody,resetBeats,RESET_MAX_COUNT);
setup();
return;
}
//let the game continue only if the game has not ended yet
if(isStart){
//If it is question and question time
if(state == QUESTION){
//Display messages showing states
if(player ==STELLA)
writemessage(" Green asks ", " Pink answers ");
else
writemessage(" Pink asks ", " Green answers ");
}
//If it is revealing answer state,
else if(state == ANSWER){
if(player==STELLA){
writemessage(" Is Pink's ", " answer correct? ");
//If Pink's answer is correct
if(digitalRead(YESJ)<1){
board[currSquare] = STELLA;
Serial.print("Pink places at position: ");
Serial.println(currSquare);
printBoard();
//play music
playMelody(successMelody,successBeats,SUCCESS_MAX_COUNT);
//Change game state to next round
currSquare = -1;
state = QUESTION;
isStart = !checkWinner();
player = !player;
}
//If Pink's answer is wrong
else if(digitalRead(NOJ)<1){
trellis.setPixelColor(currSquare, 0);
trellis.show();
//play music
playMelody(failureMelody,failureBeats,FAILURE_MAX_COUNT);
//change game state to next round
currSquare = -1;
state = QUESTION;
isStart = !checkWinner();
player = !player;
}
}
else{
writemessage(" Is Green's ", "answer correct? ");
//If Green's answer is correct
if(digitalRead(YESS)<1){
board[currSquare] = JEFFREY;
Serial.print("Green places at position: ");
Serial.println(currSquare);
printBoard();
playMelody(successMelody,successBeats,SUCCESS_MAX_COUNT);
currSquare = -1;
state = QUESTION;
isStart = !checkWinner();
player = !player;
}
//If Green's answer is wrong
else if(digitalRead(NOS)<1){
trellis.setPixelColor(currSquare, 0);
trellis.show();
playMelody(failureMelody,failureBeats,FAILURE_MAX_COUNT);
currSquare = -1;
state = QUESTION;
isStart = !checkWinner();
player = !player;
}
}
}
}
}
Chloé Desaulles
Jiatian Sun
Jianxiao Ge
Our prototype of the Automatic Filing Machine! It is made of laser-cut cardboard and wooden dowels. The tray holding the paper is intended to move up and down to the correct shelf for filing.
A close-up of our paper tray. We did not have time to make the real-life representation of it, but the papers would have a conveyor belt underneath to move the papers off of the tray and into the shelf.
A very simple representation of our user interface: a button panel. Each button would be labeled with the shelf that it goes to, and maybe color coded to aid visual recognition.
We are team MARYa (Mohan, Andrea, Roly, and Yingyang), four students at Carnegie Mellon University that are assigned with the task to create a useful implement for our older friend, Maria. Previously, we had documented our meeting with Maria to find out what could be useful for her. In this stage, we took our idea and turned it into a prototype for Maria to see and critique at our next meeting with her, which was on November 13th. We found in our initial meeting with Maria that she needed was throwing her papers on the floor instead of filing them, so we tried to come up with a product that would organize her papers for her.
Product: An Automatic Filing Cabinet
Our prototype demonstrates a cabinet that files itself. Although it is completely a “behaves-like” prototype (it has no code yet), it does represent what our team envisioned for the product.
We want Maria to be able to put papers that she needs filed onto the tray of our product, press a button, and let the machine do the rest. The machine will determine where the papers go based on the button pressed, which corresponds to a shelf. The tray will move to that shelf, and a small conveyor-belt-like mechanism will push the papers onto the shelf. The tray would then return to the top of the machine, waiting for more papers. The buttons are located on top of the machine for easy access, and the front of the shelves are open to allow the user to take the papers they need out.
Process:
The origin of our idea: Maria’s very populated filing cabinets. She has a publishing business, and thus has massive amounts of paperwork.
A visual inspiration for the proportions and materials of our shelf. We like wood as a main material, since it would fit in well with Maria’s home. We also like the strength and customizability of it.
One of our most important meetings before the critique. Here Roly is explaining his vision of the mechanism for raising and lowering the paper tray.
One of the mechanism inspirations we found during image research. We will be using a stepper motor to spin a big threaded rod, but this required attaching the stepper to it using a belt. This image shows how we will do that.
A quick drawing that we used to determine how big our shelves should be. We needed to strike a balance between accessibility for Maria and accessibility for the Machine itself.
Maria’s first glimpses of the prototype! She was very intrigued with how it would work and gave some valuable input on the materials we should use and how our interface should be positioned.
The machine in action at our prototype meeting! Maria is testing the height of our paper tray and the reachability of the buttons.
Discussion
Our first meeting with Maria was much later than the other groups’, so we were a bit behind everyone else. While the rest of the class was working on the third in-class session for their prototype, our group was meeting Maria for the first time. Because of this, we only had around 4 days to meet up and work on a prototype. We couldn’t add any of the actual motorized or mechanical features of the prototype, but we did get a full-scale cardboard version of the prototype.
The crit on November 13th was pretty useful to our group. We met with Maria, and showed her our prototype. She gave us some useful feedback that we will want to incorporate into our final product.
She asked us to put wheels under it so that it could be moved easily, which I thought was pretty useful because none of us had thought about that. We asked her feedback on the placement of the buttons, and she gave us useful feedback for that as well, saying if we put the buttons on top of the shelf she wouldn’t be tempted to put things on top of it. She also asked us to number the shelves instead of writing labels on it so that she can label them herself and change the labels if she wants to, which I thought was something we should definitely consider.
For our next steps, we have a ton of work to do. We have to order our parts, figure out the mechanical and software aspects of this, finalize our design, cut the shelf itself out, and basically everything else. It will be quite a challenge to do this in 2 weeks especially with Thanksgiving break coming up, but hopefully we will pull it off.
]]>
The full mechanism with all of the modular components shown.
The actuation is driven by a servo which turns a rack and pinion mechanism.
The servo of the switch is triggered by a push button. For the final device, the servo will be triggered by a photoresistor.
Video of device in action.
Our device turns “on” and “off” a model of Joanne’s linear toggle switch. The servo turns the gear 160 degrees and the rack meshed with the gear converts that rotation into linear actuation. A grip for the toggle switch handle is connected to the rack so when the rack moves, the switch is pushed or pulled with it. The user pushes the button to activate the servo.
Process
We first needed to take a bunch of measurements of the toggle switch at Joanne’s house.
After we had all the measurements we needed, we drew some sketches of the switch box and started generating ideas for what we were going to make to move the switch.
We decided on a rack and pinion mechanism driven by a servo for the actuation. This is the first thing we modeled in SolidWorks. The toggle switch grip is mated to the backside of the rack and we made a 4 bolt pattern on the base to attach it to the switch cover that we will also model.
We didn’t originally plan on designing a model of the toggle switch box but we decided that having a 3D model of it would make designing our mechanism easier instead of just going off a two dimensional sketch.
We mated the rack and pinion mechanism with the toggle switch box in an assembly to see where and how our device needs to attach to the box. After doing this, we realized that the 4 bolt holes on the base of our mechanism isn’t the best method of attachment.
We designed the box cover which will be the way our rack and pinion mechanism is attached to the toggle switch box. The base was removed and a segment of the lateral wall was extruded further for the 4 bolt pattern. The picture shows a full assembly of the box cover over the toggle switch box and the rack and pinion mechanism mated to the box cover. Time to print!
The print came out clean but still required a little bit of sanding to allow smooth movement of the rack and pinion. Most of the parts were connect by a screw. The picture shows a complete assembly of the device that will be attached to the toggle switch box.
Our device easily slipped onto the toggle switch box and was ready for testing.
We first uploaded some basic servo example code to test the movement of our device. We achieved the full range of linear motion that we calculated for the actual toggle switch box at Joanne’s house. After this all we had left to do was implement a way for the user to trigger the servo.
Discussion
Major CHallenges
In the previous week we gained and subsequently lost a group member. This caused a communication error and the prototype had to be given a button last minute so it could be activated during the interview. This complicated the explanation of the system to Joanne because the presence of a button implied that a button would activate a device. It caused a bigger communication error, as the most important part of the process is that a photocell inside of a front door lamp is the trigger to start the Arduino. So there is no additional button we are adding to her home, she flips the switch inside of her house and the front lamp goes on telling the Arduino to start. The presence of the button confused Joanne so it took longer to explain to her the system. The only challenge with designing our mechanism was that we did not take precise measurements of the toggle switch box at Joanne’s house so for many of our dimensions we had to estimate it. These estimations stacked up sometimes and caused certain features to be misaligned.
The Crit
Overall the crit was a good opportunity to reassure Joanne about the power source and the visibility of the wires. Her commentary mostly revolved around aesthetic of the system and concern about the dangers of the electricity. After we thoroughly explained the system and she understood exactly what was happening she was reassured. We didn’t spend much of the crit critiquing, it was mostly explaining in detail how the system would work for Joanne.
Joannes main concern was the visibility which led to a conversation about the wires and voltage. After Zack reassured her we set a voltage goal of 12 volts as we move forwards. We bonded more with Joanne and instilled a solid trust but beyond that our system is relatively straightforward and there wasn’t too much new. It reminded us of the importance of the weatherproofing but we had already been thinking about that.
We will be ignoring none of her concerns as they were all quite valid, all revolving around visibility and electricity.
Planned Next steps
We will need to guarantee the servo switch is waterproof so we are planning on finding the waterproof servo. From there we will double check with Joanne that she wants the LEDs wired into the system, she made a few comments that could have one of two meanings, concern for the visibility, and/or a desire to not have LEDs. So we will reach out to her for a decision. Aside from all of the small details we will be powering through as we have a lot of voltage and wiring problems left to tackle. Also, in order to assure that our device perfectly fits to the toggle switch we will make another trip out to Joanne’s house with a caliper and take more exact measurements.
]]>
Background
We are team MARY (Mohan, Andrea, Roly, and Yingyang) and we are currently taking an Introduction to Physical Computing course at Carnegie Mellon University. For the final project, our team was paired up with Maria, an older person who we were assigned to create a useful implement for. In order to come up with an idea for a useful implement for Maria, we first needed to talk with her face to face to face to identify areas in her life where she could use help. Unfortunately, on the week we were supposed to conduct our initial interviews, Maria was on a cruise, which provided some complications for us. We tried to conduct a FaceTime interview with her on October 30th, but that wasn’t very fruitful, especially since we couldn’t get FaceTime to work so we had to do a phone call instead. When she got back, we were able to go visit her house on November 8th and talk to her in person, which was a little more helpful than the first interview.
Meeting Agenda
(Note: the agenda below is for our first meeting over the phone)
Goal: Get to know Maria, let Maria get to know us, ask about what her life is like and how we could help her
Introductions:
- Talk about ourselves, our skills, what we want to accomplish for the project and the meeting, then ask her to do the same- what does she expect from us?
Questions:
Personal Life
- What is your daily routine like?
- What do you like doing for fun?
- Who do you live with and where?
- Frustrations in your life?
Career
- What is your job?
- What does that involve?
- If you don’t have a job, do you miss having a job?
- Do you have anything you think replaces the role of your previous job?
Hobbies
- Do you have any major hobbies/time-consuming activities?
- Is there anything you used to do, but is harder to do now?
Closing:
- Summary of what we talked about, and tell her what we plan to do until the next time we meet
Meeting Summary
Our First “Meeting”, Over the Phone
The first meeting was held over the phone because Maria was on some Arctic Cruise, and we were unable to get FaceTime to work, which I don’t think would have helped that much.
We began the meeting by introducing each of our team members and talking about some of our interests and skills. We hoped that this would give Maria a good idea of what we were able to do. We then tried to let the conversation flow into talking more about Maria’s daily life and routine to perhaps find a problem ourselves. We let the answers to each question guide us, attempting to hit major points such as her job, hobbies, and other activities.
We learned Maria writes and publishes for her publishing company, she likes to play bridge and garden, she has a son who loves Godzilla, and that she lives with her husband in Churchill.
Maria seemed to be attached to a few ideas of “gadgets” (as she called them) that she had come up with on her own beforehand, instead of letting us find a solution while she talked about her problems. She talked about how her husband leaves their kitchen cabinets open and it frustrates her, but that problem is not necessarily an electronics/Arduino problem. She also discussed her and her friends losing at bridge against her husband, and how a dispenser of chocolate can be made for them, but this is again not a very substantial problem to solve. It was very hard to get her to stray from these ideas.
Overall, this meeting was not extremely conducive to ideation, since the team was not able to get an idea to work off of.
A photo of us struggling to get FaceTime to work for our first “meeting” with Maria
Meeting Maria in Person and Going to Her House
When Maria returned to Pittsburgh from her cruise, we were able to set up a date for her to drive us to her home (November 8th). This meeting was critical because we had come up with absolutely nothing from the first meeting.
Maria first drove us to her new home, which was still being built. We did not find this particularly interesting or helpful, but Maria was pretty set on showing us her new home. We were more interested in her current living situation to see how she lives and goes about her day; so after a lot of insistence on our part to go to her current home, she finally took us there.
This meeting was more fruitful than the first, but we still couldn’t identify major problem points that we could solve with an Arduino that wasn’t a product that already exists. We also got to see her husband leaving the cabinet doors open, which was fun.
We got to see Maria’s workspace, and it was quite cluttered. There was paper everywhere and in piles. Maria has a publishing company, so this was understandable. However, we learned that Maria has a habit of throwing her papers on the floor and not filing or sorting them, which makes it hard for her to remember what papers she put where. It is important to keep track of where she puts things because she has so many papers to deal with, and throwing them on the floor also makes them hard to reach. Maria has a file cabinet and those desktop paper organizers with the different sections, but she still throws them on the floor anyways.
Although this is more of a laziness/bad habit problem instead of something she is physically unable or difficult to do, it gave us at least something to work with.
Maria driving us to her house
Maria showing us her new home that is being built
Maria’s very cluttered desktop
Finally getting where we want to go
So much paperwork and clutter
She doesn’t remember where she filed her papers away
Maria making some hand gestures
Giving us a tour of her space
Reflection
The first meeting was kind of terrible since it provided no material for us to work with, but the second one provided us with at least something.
For our first conversation, we were frustrated that Maria kept pushing ideas for “gadgets” that she’d come up with instead of letting us get to know her and the difficulties she faces in her daily life. Perhaps we had asked the wrong questions. We thought they were generic enough, but it is difficult when she had already planned out what she wanted to talk about, instead of letting our questions guide her.
It was also difficult for us to get her to understand that we wanted to see her current home, not her new one. When we are faced with a blank space that she doesn’t live in yet, we are unable to see what she is like at home, how she goes about her life, and interacts with her environment.
]]>For the final project of Introduction to Physical Computing, The Weavers, which consists of Jenny, Megan, and Ghalya, was assigned to create an assistive device for Rebecca Herbert. Rebecca loves weaving. For her, the most tedious task in weaving is winding the yarn into a ball because it hurts her wrist and she cannot tell how much she winded. This documents the initial prototype of a motorized ball winder that would save Rebecca from doing it herself.
PRODUCT
Ball winder in action.
Full shot of prototype.
User setting the amount of yarn that the ball winder will wind.
We created an automatic ball winder; when weaving, the weaver must initially take the yarn they have and turn it into a form that holds its shape well and is easy to use. Most people do this using a ball winder, which requires the user to turn a knob around until the yarn has gotten into a ball. This step can take Rebecca hours, leading her to get greater pain in her arms, and discouraging her from starting new projects. We have set out to create a ball winder that automatically winds the amount of yarn that the user wants without having to manually turn a crank.
PROCESS PHOTOS
This is a ball winder that Rebecca uses regularly to ball her yarn by hand; we used it as a guide on how to make an automatic version.
These are two of the original schematics we made for our ball winder; it also shows some of our preliminary circuitry.
Testing Individual Components of Prototype
Top Surface of the Device with Prototype String Feeders
Diagonal string winding pattern of the prototype.
Testing different heights of to feed the string into the prototype.
Prototype spinning while collecting yarn.
This shows what our ball winder looked like when in use with Rebecca’s umbrella; this is how it would be used realistically in her house.
This is a video where we tried comparing the movement of our ball winder with the movement of the sample one Rebecca has to see how it moves differently.
Finished Display Screen
Our crit and talk with Rebecca.
DISCUSSION
Initially, the major problems were trying to figure out how to work the LED screen and the motor; we spent a while working on them and wound up going to office hours for help. After we assembled all the parts, we discovered that the yarn pattern on our ball winder is wrong; instead of forming a criss-cross shape that holds its form well, our ball winder creates a diagonal shape where the yarn goes over itself and doesn’t hold its shape. We spent a while testing different reasons why this may be by holding the yarn at different heights, changing the speed, and extensively comparing our design with the design of the manual ball winder we were given. We were told by Rebecca that the yarn pattern is not the most important step- so going forward we will work on perfecting our distance measuring before focusing on the yarn pattern more.
The critique on November 13th was extremely helpful in the process because it allowed us to touch base again with what Rebecca envisioned for her device. As we worked on the prototype, the string winding pattern was particularly challenging for us and we spent most of the time trying to mimic the exact winding pattern of a manual ball winder. However, after the critique and Rebecca’s opinion, we realized that the most important thing to her was to find the length of the yarn wound on the ball, not the yarn pattern of the ball. While the yarn pattern is helpful in keeping the shape of the ball after taking it off the ball winder, it is not essential to the project. Rebecca also reminisced that her grandmother would wind yarn around her hands or between a person’s arms, which results in the same winding pattern as our prototype currently has. Thus, moving forward we will be focusing mainly on calculating an accurate distance that the yarn was wound, and try to solve the yarn pattern if time allows.
Another thing that we realized during the critique is that the design of the automatic ball winder must be more user-friendly. During the critique it did not function as intended initially due to some wiring coming undone; we had to make sure that it was plugged in. The button that we were pressing on the rotary encoder wasn’t easily pressed when placed in the box as well, so that needs to be fixed. In general, we want it to be easier for the user to input the amount of yarn they want to wind and to set up the ball winder.
The next steps of this project would be to implement accurate distance measuring functionality alongside a intuitive user design. This would entail measuring the string on a wheel before the yarn is put into the ball form. The user would loop the yarn into a feeder hole which gives the yarn to a rotating disk with a set size. The number of rotations is used to calculate the total distance traveled and therefore the length of the yarn. Then, the thread is fed through another feeder hole and wound into the ball. Currently, there is a screen with a dial that the user can use to input the amount of yarn they would like and press the dial starts the winding process. The screen also alerts the user how many yards are have currently been wound. To make the process easier and to give more functionality for emergency stops or different settings, a keypad would offer more options for the user to have. The main focus for the future of this project is to gain an accurate reading of the yarn measured as well as make the process of using the product as streamlined as possible.
]]>