The goal of this project was to create a specialized keyboard and mouse for our design client, Kim, to make gaming on her computer more accessible. We first interviewed Kim to understand her daily routine and challenges. From there we ideated different assistive devices to address these daily problems. We had learned that Kim has a physical disability that limits the use of her right hand so she depends solely on her left hand when playing games on her laptop. Kim can grasp objects with her right hand but needs the assistance of her left hand to release her grip. We wanted to address this challenge by designing a keyboard catered to her left hand and a joystick that has a release mechanism for her right hand. Hence, we built this gaming device catered to her needs and preferences. More information about our interview with Kim can be found here: https://courses.ideate.cmu.edu/60-223/s2022/work/team-dione-interview-documentations/

Final Product

a. Description

Our assistive Gaming Keyboard and Mouse device was designed to make gaming on the computer more accessible to our client Kim. The keyboard was created to Kim’s personal preferences and controls keys on the commonly used in gaming. Its interface inspired by classic arcade machines with the goal of making the gaming experience more enjoyable for Kim. We experimented with different layouts and shapes for the keyboard and finalized on a design that had optimal comfort and usability for Kim. The overall goal for this keyboard was to make the buttons used to play games more accessible for Kim’s left hand. In addition to the keyboard we also designed a mouse for Kim’s right hand. The mouse is a 3D printed joystick that controls the cursor and it has an inflatable plastic bag around it which helps our client release and firmly grasp the joystick.

b. Images

Front view of the device

start game by using joystick and playing snake game with keyboard

perspective view of the arcade keyboard

perspective view of the inflatable joystick

side view of the joystick

detail view of the convex 3d printed shell and led buttons

c. Narrative Sketch

When Kim wants to play a game on her laptop, she plugs in the USB-C cable into her laptop and presses the on switch. She knows the gaming device is up and running once the LED arcade buttons light up. She then controls the joystick with her right hand and uses the keyboard with her left hand. When she wants to release her hand from the joystick she presses the bottom left button on the keyboard. This button activates the air pump which pumps air into the joystick for 20 seconds. Once the joystick is inflated, Kim is able to easily remove her hand from the joystick. Then when Kim wants to use the joystick again she can press the bottom left button on the keyboard to deflate it. When Kim is done playing games she presses the switch off and unplugs the USB-C cable from her laptop.

Prototype and Process

This prototype was created to answer the design question of how each feature meets Kim’s needs and how we should change it to her convenience. The goal for our prototype was to create a works like prototype, focusing on the electronics and individual parts of the system performing the mechanisms we ideated.

Our prototype consisted of three different features, the joystick, the keyboard and the inflatable mechanism. The joystick was a functioning joystick that controlled the laptop cursor, the keyboard focused on the placement of the buttons and the ease of use for Kim and the inflatable mechanism tested its functionality for Kim’s use.

This is the first iteration of the keyboard. The labels show the letters the buttons were originally mapped to test and debug

wiring and testing the joystick on the Arduino micro

back side wiring for the arcade button keyboard

The prototype was effective in identifying areas of improvement for our design. Our initial keyboard iteration had a leveled surface where all the buttons laid flat. We had Kim test this design by playing a game so we can observe what adjustments to make. Kim was elated to see the colorful arcade buttons however her discomfort was evident when trying to navigate the buttons. She struggled with where to place her hand and some buttons seemed inconvenient to reach. Moving forward we wanted to experiment with the keyboard’s ergonomics to optimize comfort and usability for Kim. From Kim’s feedback we redesigned the keyboard to have a curvature on its surface and changed the layout of the buttons.

The joystick we used for our prototype was a small Arduino joystick from the physical computing lab. Our goal was to get the code and joystick to work for the prototype so moving forward we could experiment with its size, shape, and orientation. The prototype was functional in its control over the laptop. We could not receive much feedback on its current design from Kim since it was a functional prototype. However, we did have Kim grasp different sized objects with her right hand so we could gain a better understanding on the ideal shape for the joystick.

The third component to our prototype was the inflating mechanism. We designed a separate device to demonstrate the inflating and deflating concept for the joystick. This device consisted of an air pump, tubing, and a balloon. Kim tested if this mechanism was effective in releasing her hand so we could understand what size and shape would be ideal moving forward. From our observation and Kim’s feedback the balloon was effective but the air pump was too slow. Moving forward we sought out different air pumps that could pump air at a faster rate and experimented with more durable materials than a balloon.

figuring how to transit to a larger air pump

testing the fitting of airbag and joystick extension

This image shows all of the wires that had to be connected to the Arduino. We used tape to label the pin each wire should be connected to keep the wires organize in case they became detached.

This diagram represents the layout of the buttons and the number we assigned to each one of them to stay organized during the debugging process.

The two images above were diagrams we created for the debugging process. We created a system to test each button and ensure its code and wiring was correct

One of the final iterations o the keyboard with the convex surface prior to being painted.

testing the new air pump with the Arduino uno

The early prototyping stages of this project were focused on creating a device that works. After we developed a functioning keyboard, air pump and joystick we shifted our focus to the ergonomics. After the prototype critique and user testing we had many new ideas on how to improve the overall design to increase comfort and accessibility. We wanted to experiment with the size and shape of the keyboard. We drew inspiration from different keyboards we saw online and created a new model with a curved surface. In addition to redesigning the keyboard, we wanted to find a more durable joystick to use. With a more durable joy-stick we were able to model a 3-D printed shell personalized to Kim’s preferences that was easier for her to grip. Throughout the redesign process we had many conversations with Kim to receive more feedback on these new ideas. This bore fruit as we added contour lines on it to ensure that her hand does not slide off and we decided to include the factor of visual appeal to the keyboard.

To improve the user interface of the device we wanted to add a visual indicator to signal to the use the device is on. Our initial idea was to add an LED strip around the perimeter of the keyboard that would light up when Kim switched the device on. Then we learned there are arcade buttons that had LED lights we decided to move forward with these instead. In our final iteration of the project we programmed the LED buttons to light up when the device is on.

The inflatable mechanism went through multiple revisions. After the prototype critique we wanted to create a more durable and faster inflatable mechanism. We considered using water instead of air to pump it but decided air would be a wiser choice since we are dealing with electronics. We consulted Professor Zach about experimenting with other materials instead of a balloon such as casting a silicon bag which would be soft and would avoid issues where the air leaks out. To pursue this option, we met with Professor Garth and spent a class discussing this with him. However, he said the main difficulty would be to cast the silicon bag in the shape we wanted as the silicon gel would have to be constantly rotated in all directions to create our preferred shape. He suggested we adopt any existing product that serves the same purpose and this helped us in fixating on inflatable plastic bags that have a compartment with them. This was perfect as it properly wrapped around the joystick and inflated to the required volume for Kim’s needs.

There were a couple of issues during the development of the final product. Our first 3d print of the curved surface was not strong and was disintegrating so, we printed another with a thicker width to ensure structural integrity. One of the potentiometers of the joystick broke so only one axis worked for the joystick. We had to buy another one post the final presentation. Our schedule was delayed by a few days due to the revisions in our design as there were products like the joystick and the buttons that had to be ordered online. A few days before the final presentation, the micro-Arduino’s port broke while we were testing it which was stressful and we required a replacement.

Gantt Tracking 

This is the Gantt chart we used to plan and track our progress throughout the assignment. Even though we tried to follow it diligiently, we could not plan for certain problems that occurred. Some unforeseen challenges like the micro Arduino breaking set us behind schedule and the 3-D print deforming. However, it was useful to have this schedule to reference to make sure we were maximizing our work time and knew what tasks to complete next.

Conclusions

The final critique provided us with a lot of valuable feedback and new perspectives on the direction and potential for the mouse and gaming machine. When we first sketched the inflatable pump, we were all skeptical about its feasibility. In the end, we were happy to develop a mechanism that accomplished what we imagined. Even though this idea shows promise there is a lot of potential for improvement. One common issue that was raised about the pneumatic pump bag was “Pneumatics are slightly leaky” and “Pneumatic pump/bag on lever fragile to pinholes”. We were concerned about the longevity of the air bag and the pipe connections into the air bag but we opted to proceed with it to ensure the completion of a workable project. However, we did note that for further considerations of the project, we would explore other materials and better pneumatic fittings for the inflatable mechanism.

We spent the latter of this project working to improve the keyboard’s design, specifically its shape and layout. The feedback from the critique opened our eyes to other aspects of the design that can be refined. One person mentioned that the wires that connected the joystick and keyboard looked messy. In the future we would hope to have the keyboard and joystick be connected wirelessly. A wireless connection would improve the aesthetic and usability of the gaming device. Another piece of criticism we received was about the lack of labels on the buttons. Including labels would improve the user interface of the keyboard. Another design critique was that, “The clear box is a little messy with acrylic cement”. One of the critiquers gave us feedback with regard to the motive of using a clear acrylic box for the mount of the joystick and our rationale for glueing it together with acrylic cement instead of screws. We chose the acrylic box to display the inner mechanism which added to the overall aesthetic of the project however, the acrylic cement smudged the clarity of the acryclic build in certain places. This would be an area of improvement for further versions of the same project.

Designing for other people is a very rewarding experience. We all enjoyed working with Kim to create a device she was excited to use and could help her in her everyday life. Kim may have physical limitations but she does not allow it to inhibit her lifestyle. She has so many hobbies and interests I think we all felt inspired to do more of what we love after talking to her. It felt so special to have the opportunity to create a device that can make Kim’s hobbies and interests more accessible to her. The world is not designed for people with disabilities so it was rewarding to make Kim something that was for her. Seeing Kim smile and laugh using the gaming device made all the work we put in worth it.

The timeline of this project gave each of us exposure to the product design process. From the interview to the final product it is rewarding to see how much we accomplished in a short amount of time. For some of us, it was our first experience designing for a client. It was challenging at times to design for some one else when they are not present. We had to make decisions that we believed were in our client’s best interest but sometimes it felt like a guessing game. The feedback we did receive from Kim gave us direction and propelled us forward throughout the project. It felt like most of our ideas were hers and it was our responsibility to make them come to life. Compared to the last assignment the pressure of designing for a client weighs much heavier than designing for yourself. We not only wanted to deliver a working project but also a project that met Kim’s expectations. In the end we were able to develop a product that our client enjoyed using however with more user testing and feedback it could be improved.

Technical details

Schematic and block diagram

Code

• the project title,
• (optionally) your names,
• a description (short or long) of what the code does,
• any description of pin mapping that would be useful to somebody else trying to recreate your work,
• appropriate credit to any code source that you incorporated into your project, and
• (optionally) a license notice (i.e. copyright, CC BY-SA 4.0, the MIT License, release it to the public domain, or just follow your heart). If you have written code that you wish to keep strictly proprietary for any reason, please speak with the instructor about an exception to this documentation requirement.

//Gaming Keyboard and Mouse 

//This code controls the air pump, keyboard, and joystick. When the user turns the switch on the LED buttons turn on. There are 9 buttons on the keyboard and the code maps each of the buttons to a key. When the code reads the buttons as high then the code presses the key it is mapped to. After the button is done being pressed the code then reads the button pin as low and releases the key that was being pressed. This ensures that the key is only pressed once when the button is pressed once. The code for the joystick takes in the inputs from the potentiometer along the x and y axis and outputs a direction the computer mouse should move in. The code for the air pump inflates air when the bottom left button on the keyboard is pressed. The next time this button is pressed the air pump deflates the air bag surrounding the joystick. 

/*
Pin Mapping:
Arduino Micro Pin | type   | description 
------------------|--------|------------
A1                  Output   LED

A2                  Input    Joystick X axis

A3                  Input    Joystick Y axis

2                   Input    Switch(controlled whether the keyboard was on or off)

3                   Output   Pump

4                   Output   Pump

5                   Input    Button(controls shift key)

6                   Input    Button(controls space key)

7                   Input    Button(controls right arrow key)

8                   Input    Button(controls return key)

9                   Input    Button(controls up key)

10                  Input    Button(controls down arrow key)

14                  Input    Button(controls left arrow key)

15                  Input    Button(controls delete key)

16                  Input    Button(controls if pump is on or off)


*/


//https://docs.arduino.cc/tutorials/micro/keyboard-press


#include <Keyboard.h>
#include <Mouse.h>

//on button
//switch
const int onButton = 2;

//buttons 
//LED BUTTONS
const int upButton = 9;
int previousupButtonState = LOW;
const int downButton = 10;
int previousdownButtonState = LOW;
const int rightButton = 7;
int previousrightButtonState = LOW;
const int leftButton = 14;
int previousleftButtonState = LOW;
const int spaceButton = 6;
int previousspaceButtonState = LOW;
const int shiftButton = 5;
int previousshiftButtonState = HIGH;
const int returnButton = 8;
int previousreturnButtonState = HIGH;
const int deleteButton = 15;
int previousdeleteButtonState = HIGH;

//LED
const int LED = A1;


//constants for air pump control
const int pumpButton = 16;
int buttonState = HIGH;//pump input
const int PUMP1 = 3;
const int PUMP2 = 4;
bool pressVal = false;
bool AIRIN = 0;
bool pumpOn = false;
unsigned long timecount = 0;//this is to check whether the pump is on Periodically

//Constants for joystick control
//  pin numbers for switch, joystick axes, and LED:
const int xAxis = A2;         // joystick X axis
const int yAxis = A3;         // joystick Y axis
// parameters for reading the joystick:
int range = 12;               // output range of X or Y movement
int responseDelay = 5;        // response delay of the mouse, in ms
int threshold = range / 4;    // resting threshold
int center = range / 2;       // resting position value
 // whether or not to control the mouse
bool mouseIsActive = false;   
int lastSwitchState = LOW;   









void setup() {
   pinMode(onButton, INPUT); 
   pinMode(LED, OUTPUT);  
   pinMode(upButton, INPUT_PULLUP);   
   pinMode(downButton, INPUT_PULLUP); 
   pinMode(leftButton, INPUT_PULLUP); 
   pinMode(rightButton, INPUT_PULLUP); 
   pinMode(spaceButton, INPUT_PULLUP);   
   pinMode(shiftButton, INPUT_PULLUP); 
   pinMode(returnButton, INPUT_PULLUP); 
   pinMode(deleteButton, INPUT_PULLUP);  
   pinMode(pumpButton, INPUT_PULLUP);
   pinMode(PUMP1, OUTPUT); 
   pinMode(PUMP1, OUTPUT);
   
   Mouse.begin();   
   Keyboard.begin();
   Serial.begin(9600);
}

void loop() {
  //code for the keyboard
//Serial.println(" working");
Serial.println(digitalRead(onButton));
if (digitalRead(onButton) == HIGH){
  digitalWrite(LED, HIGH);
//upbutton:218/UP
  int upbuttonState = digitalRead(upButton);
   //replaces button press with UP arrow  
  if (upbuttonState == HIGH && previousupButtonState == LOW) {      
     Keyboard.press(218); 
     //delay(50);
   }
  if (upbuttonState == LOW && previousupButtonState == HIGH) {
    Keyboard.release(218);
    delay(50);
  }
   previousupButtonState = upbuttonState;
   
//downbutton:217/DOWN
  int downbuttonState = digitalRead(downButton); 
   if (downbuttonState == HIGH && previousdownButtonState == LOW) {      
     Keyboard.press(217); 
     delay(50);
   }
  if (downbuttonState == LOW && previousdownButtonState == HIGH) {
    Keyboard.release(217);
    delay(50);
  }
   previousdownButtonState = downbuttonState;

//right button:215/RIGHT
  int rightbuttonState = digitalRead(rightButton);
   if (rightbuttonState == HIGH && previousrightButtonState == LOW) {      
     Keyboard.press(215); 
     delay(50);
   }
  if (rightbuttonState == LOW && previousrightButtonState == HIGH) {
    Keyboard.release(215);
    delay(50);
  }
   previousrightButtonState = rightbuttonState;

//leftbutton:216/LEFT
  int leftbuttonState = digitalRead(leftButton);
   if (leftbuttonState == HIGH && previousleftButtonState == LOW) {      
     Keyboard.press(216); 
     delay(50);
   }
  if (leftbuttonState == LOW && previousleftButtonState == HIGH) {
    Keyboard.release(216);
    delay(50);
  }
   previousleftButtonState = leftbuttonState;
   

//spacebutton: 32
  int spacebuttonState = digitalRead(spaceButton);
   if (spacebuttonState == HIGH && previousspaceButtonState == LOW) {      
     Keyboard.press(32); 
     delay(50);
   }
  if (spacebuttonState == LOW && previousspaceButtonState == HIGH) {
    Keyboard.release(32);
    delay(50);
  }
   previousspaceButtonState = spacebuttonState;

//shiftbutton:224
  int shiftbuttonState = digitalRead(shiftButton);
   if (shiftbuttonState == LOW && previousshiftButtonState == HIGH) {      
     Keyboard.press(224); 
     delay(50);
   }
  if (shiftbuttonState == HIGH && previousshiftButtonState == LOW) {
    Keyboard.release(224);
    delay(50);
  }
   previousshiftButtonState = shiftbuttonState;


//returnbutton:176
  int returnbuttonState = digitalRead(returnButton);
   if (returnbuttonState == LOW && previousreturnButtonState == HIGH) {      
     Keyboard.press(176); 
     delay(50);
   }
  if (returnbuttonState == HIGH && previousreturnButtonState == LOW) {
    Keyboard.release(176);
    delay(50);
  }
   previousreturnButtonState = returnbuttonState;


//deletebutton:BACKSPACE(8)
  int deletebuttonState = digitalRead(deleteButton);
   if (deletebuttonState == LOW && previousdeleteButtonState == HIGH) {      
     Keyboard.press(8); 
     delay(50);
   }
  if (deletebuttonState == HIGH && previousdeleteButtonState == LOW) {
    Keyboard.release(8);
    delay(50);
  }
   previousdeleteButtonState = deletebuttonState;

}
if (digitalRead(onButton) == LOW){
  digitalWrite(LED, LOW);
}




//code for pump
  buttonState = digitalRead(pumpButton);
  if (pumpOn == true){
    if (millis()-timecount > 30000){
      pumpOn = false;
      digitalWrite(PUMP1,LOW);//turn motor off
      digitalWrite(PUMP2,LOW);
    }
  }
  if (buttonState == LOW ){
    Serial.println("the pumpbutton is pressed");
    timecount = millis();
    pumpOn = true;
    if(pressVal == false){
      pressVal = true;
      digitalWrite(PUMP1,HIGH);//turn motor on, starts inflating
      digitalWrite(PUMP2,LOW);
    } 
    else {
      pressVal = false;
      digitalWrite(PUMP1,LOW);//turn motor on, starts deflating
      digitalWrite(PUMP2,HIGH);
    }
  }

//Code for the joystick

  int switchState = digitalRead(onButton);
  // if it's changed and it's high, toggle the mouse state:
  if (switchState != lastSwitchState) {
    if (switchState == HIGH) {
      mouseIsActive = !mouseIsActive;
    }
  }
  // save switch state for next comparison:
  lastSwitchState = switchState;

  // read and scale the two axes:
  int xReading = readAxis(xAxis);
  int yReading = readAxis(yAxis);

  // if the mouse control state is active, move the mouse:
  if (mouseIsActive) {
    Mouse.move(xReading, yReading, 0);
  } else { 
     //else the mouse button is not pressed:
    // if the mouse is pressed, release it:
    if (Mouse.isPressed(MOUSE_LEFT)) {
      Mouse.release(MOUSE_LEFT);
    }
  }

  delay(50);


}// end of the code

int readAxis(int thisAxis) {
  // read the analog input:
  int reading = analogRead(thisAxis);

  // map the reading from the analog input range to the output range:
  reading = map(reading, 0, 1023, 0, range);

  // if the output reading is outside from the rest position threshold, use it:
  int distance = reading - center;

  if (abs(distance) < threshold) {
    distance = 0;
  }

  // return the distance for this axis:
  return distance;
}

The project objective is to design an assistive device for our client with physical difficulties. On Mar. 21, 2022, we held a short interview with our client, Haleigh, on the ground floor of Hunt Library; the talk gave us a chance to better understand Haleigh’s daily life, interests, difficulties, and some of her expectations towards a device that could make her life colorful. We also discussed some of our initial ideas and dived into details that specifically fit our client’s needs.

Interview With Haleigh

Agenda

1. Greetings and Self-introduction

We started our interview with a short self-introduction. It’s not in a very formal form as we already know each other in the project launch class in the previous week. We basically start with a short greeting outside the Hunt Library, talk about some activities before coming to this interview, and walk together into the open classroom located on the ground floor.

2. Project Introduction

After sitting down, we provided an overview of this project; we stated that our goal is to make a prototype of an assistant device for Haleigh within the 6-week timeframe; we also clarified that the project is an explorative one rather than a well-crafted commercial one. Before starting the interview, we asked our client and her mom if we could record our conversation and take photos during our conversion, and they agreed.

3. Understanding Needs

We started the interview by talking about our client’s routine daily life. We acknowledged that she’s interested in painting and reading during the conversation, and she also likes the wordle game. Our client and her mom also told us there are many difficulties in everyday life, like lacking grip and twisting on her hand, limited vision, and needing help when eating due to the posture problem. One of our members also holds Haleigh’s hand under consent to see how much grip she has.

4. Random Chat

The fourth part of the interview is a random chat. We don’t have preset topics but just started with a general question of what was a memorable thing in her life. The conversation was going freely; everyone on the table joined the talk and shared many exciting things like culture, favorite weather, and daily life, not just Haleigh talking and we listening aside. The random chat made us understand our client better and gave us a more vivid image of our client. There are occasional silent moments during the conversation, but we all tried to bring up new topics and make the talk continue.

5. Conclusion and Appreciation

The conversation lasted about 45 minutes. We appreciated our client’s time coming for this interview and her patience in answering our questions. Haleigh also told us she’s happy that have someone listening to her. In the end, we told them that instead of doing a one-time interview, we regard this meeting as a starting point so that we can keep in touch with our client through text when we have ideas or further questions. And we will send her our product at the end of this semester if possible.

Interview Notes

Summary / Takeaways

The interview provided us with a better understanding of our client; we got to know her daily life and interests and found some important points related to Haleigh’s difficulty that repeatedly appeared during the conversation.

1. Grip issue

The grip issue, and twist issue on Haleigh’s two hands cause one of the major inconveniences in her everyday life. She couldn’t drink easily because of lack of grip; we found it was her mom holding the cup for her and using a straw for drinking water. Our client also mentioned she likes pizza and steak, but she couldn’t cut them because of the grip issue and some twisting difficulties on her wrist; that means carrying and using any utensils is a heavy task for our clients. The same issue also limited her chance to work on her interests like painting, due to the problem of holding pens, brushes, or some similar tools.

Hearing this, we discussed the possibility of creating a device that integrates daily tools like utensils; it doesn’t need a lot of grip or precise gestures to use. One of our members held both of Haleigh’s hands under her and her mom’s consent and found that the amount of grip was good. Haleigh’s mom also showed us the gesture that Haleigh holds things most easily, and we took a photo of that pose. Thus we have empirical information on Haleigh’s grip and twist issue and enable us to work toward our client’s specific needs.

Gesture of Holding Things

Untensil Device Sketch

2. Skin issue

When talking about our favorite weather, our client said that she likes sunny days with a smile on her face. She doesn’t like cloudy days because she always like to stay outside and enjoy the sunshine and fresh air. But her mom mentioned that she couldn’t stay outside too long becase the solar radiation would hurt her skin and harm her eyes. We noticed that when hearing that the sunlight was “bad” to her, Haleigh’s face turned a little sad. We don’t want her deprived of the chance to enjoy the great weather and outdoor activities, so we arose another idea of designing a sun angle responsive “umbrella” on our client’s chair that can block the solar radiation when staying outside. This might need heavier engineering, but we kept it as one of our project directions.

Responsive Shading Sketch

Reflection

The meeting followed the agenda very well, we started the conversation smoothly and it went pretty well. Although there were some silence in that short 45 minutes, we all tried to drop new topics and keep the conversation going. Our client was so nice, trying to answer our questions in detail and sharing many exciting moments in her life. We are also impressed by Haleigh’s optimism and shinny smiles.

The interview was also productive; we got important information from our client like her interests, difficulties, and some expectations. With this knowledge, we can design a device that meets her needs better.

This is the documentation for the two interviews our team, team Dione, conducted with our client Kim and her father Bob. The purpose of this interview was to understand the Kim’s habits, hobbies and uncover any daily challenges she experiences that we could potentially solve. After we compiled a list of challenges, we devised ways to address them with assistive devices. Our goal was to gain Kim’s perspective on what it’s like living with a disability, how it impacts her daily routine and better understand her relationship with her assertive devices. With this in mind, we were able to ideate devices that Kim would be excited to use and that could make a difference in her every day life.

Meeting Agenda

The first meeting on March 21st includes basic understanding of client’s interests, needs and preference. We’ve prepared the following list of questions to ask:

• Introductions
  • Each team member introduce themselves
  • Have Kim introduce herself
• Living habits
  • Describe a typical day in your life
  • What parts of your day do you look forward to?
  • Are there any parts of your daily routine do you dread doing?
• Hobbies/Interests/Favorite Activities
  • How do you like to spend your free time?
  • What are some of your hobbies?
    • Are there any challenges or frustrations you face when doing these hobbies?
  • What are some of your interests?
    • Favorite foods, color, movies, books, games
    • Are there any problems that prevent you from pursuing any of these interests?
  • What extracurricular activities are you apart of?
  • Are there any hobbies or activities that you’ve always wanted to try but haven’t had the chance?

The second meeting on March 25th involves better understanding of client’s physical ability through demonstrations and artifact representations. It is to clarify confusion and misunderstanding in the first interview. Prior to the meeting we’ve prepare to ask Kim to perform the following task:

• To paint with Acrylic on Canvas to see how she uses brushes, how her father assist her during the process
• Play games on laptop or type on keyboard to show what keys and functions she usually uses

Questions for meeting 2:

• Questions about Kim’s experience using a keyboard
  • What games do you like playing on your laptop?
  • Which keys do you typically use on the laptop?
  • Aside from games, what else do you use your laptop for?
  • Do you use the mouse or the track pad?
  • Do you face any discomfort when typing on the key board or when using the mouse?
  • Do you use the keyboard to type our essays, letters, or emails?
• Questions about Kim’s experience with arts and crafts, specifically painting
  • How do you set up your art supplies to paint?
  • Do you paint at home or only in class?
    • What prevents you from painting at home?
  • What discomfort or challenges do you face when painting?
  • How do you clean up your art supplies?
  • How do you situate the canvas?

Summary And Documentations

The following text are points summarizing from the interview:

Most of our conversation with Kim focused on her love for painting and playing video games on her computer. From these conversations we learned Kim has limited motion in her right arm, she cannot bend her elbow and her hand can only grasp lightly but could not release. Due to these restraints, Kim heavily depends on her left hand when painting and playing video games. Kim and her father were excited by the potential to create a device that can help her use her right hand again. However, after learning about Kim’s extensive physical therapy journey to try and regain motor control over her right hand, we pivoted from that idea. Instead, we looked at the potential to create a device to help Kim use her left hand more efficiently and decrease Kim’s dependence on others when doing her hobbies.

Kim used to be righthanded but she is skilled with her left hand in craft, typing and writing, through our observation during the second Interview. In the first Interview we became acquainted with Kim’s lifestyle, she told us about her interests, hobbies and weekly routine. These are the notes we wrote down during the interview:

• Kim is enrolled in a cooking class
  • Enjoys making desserts
    • Loves the make and decorate cakes
    • Challenges
      • Cannot cut, slice, or dice due to the lack of control in her right hand
      • Maneuvering around the kitchen-kitchen layout
  • Helps out her Dad when he is cooking
    • They enjoy making pizza together
• Kim is in an art class
  • Enjoys Painting
    • Art class has special supplies that is catered towards artists with disabilities
    • Usually paints on a canvas laying flat on a table
      • Wants to try painting on a canvas when it’s on an easel
    • What Kim likes to paint
      • Flowers
      • Horses
      • Buildings
      • Landscape
      • Looking at the picture and draw on it
    • Challenges
      • Setting up the supplies requires assistance
      • Cleaning the brushes
      • Changing the dirty water out with clean water
      • positioning the canvas
      • Sometimes hard time to come up with inspiration
  • Enjoys pottery
  • A painting on the office
  • Likes to make arts and crafts
    • Crafts: necklace, bracelet
    • made a small miniature design of a her dream bedroom for her art class
    • Purse: beads hanging on the purse
• In a computer class
  • Loves playing games on her computer and tablet 
    • Candy crash
    • Card games
      • solitare
      • wants to play with physical cards but has not tried that yet
    • Bigger computers desktop
    • Tablet at home
      • Special equipment for placing the tablets
• Her school takes weekly field trips that she enjoys 
  • Movies
  • zoo
• Interests
  • Favorite movies: titanic, Mamamia Cinderella
  • Favorite music: country music, soft rocks
  • Loves the color red 
• Kim does therapy horse back riding every week
  • They have specialized equipment to get her from her wheelchair onto the horse
  • Kim loves horses

Kim’s Acrylic Painting during Interview

The way Kim is holding the brush with her left hand

Kim’s pottery with paint

Kim’s bedroom miniature that she made by herself

Kim’s painting of Titanic(her favorite movie) during her art class

Bracelet made by Kim

Discussions and Ideation Reviews

Kim enjoys spending her free time playing video games on her tablet and computer. When we learned Kim has limited motor control over her right hand we were inquisitive about her experience using a keyboard with one hand. The layout of the keyboard is designed for users to use with two hands. We thought there was potential to create a personalized keyboard for Kim to use when playing video games on the computer. This keyboard would have a layout specialized for only using your left hand and would only include keys that Kim uses when gaming. In the second interview, we pitched this design to Kim to gauge whether she would want or use this device. After pitching the specialized keyboard to Kim we learned that she doesn’t really use the keys when gaming on her computer since her computer is touch screen. She was still interested in the concept and the potential of using keys rather than touch screen.

The second design was inspired by Kim’s passion for painting. Kim described how much she enjoys painting in her art class and even demonstrated painting to us during the second interview. From our observation, Kim needs assistance setting up the painting supplies, changing out the water to clean the brushes, and cleaning up the supplies. Kim also struggles with cleaning the brushes and keeping the supplies secure since she can only use one hand. After observing these challenges Kim faces when painting, we began to devise ways we could mitigate them. We developed some ideas to create an electronic paint pallet that could be controlled by Kim with a set of buttons. The pallet can be rotated by Kim using a joystick(the pallet has LEDs next to each crevice that light up has it is being turned). The pallet can expel dirty water and refill with clean water from a tank of water stored within the device. The pallet has holes to attach each paint tube underneath and seal the paint. Kim can control how much paint is dispensed onto the pallet using the set of control buttons. When Kim rotates the pallet using the joystick and chooses the paint tube she wants to dispense(using the joystick and the selection is indicated by which LED lights up) she presses a button to confirm the selection. Once she confirms the selection, the desired paint is dispensed into the pallet. These are just some of the original features we came up with to assist Kim while painting. There is potential to come up with other ways to make painting a more independent task for Kim.

Team Callisto had the opportunity to speak with Mary D’Ottavio to design an assistive device for Mary’s daily life. Through the Ideate: Physical Computing course at CMU we were able to meet with Mary and ask her about how she navigates her life with her physical disabilities. The Team consisting of Sumayya, Yongwen, Juan and Tristan held multiple interviews with Mary to really understand what type of device would benefit her. On Wednesday, March 23 Mary told the team about her love for sports, writing, card collecting and emphasized some issues regarding accessing items that are far out of reach. After some brainstorming, the team decided to meet with Mary again on March 27 to go over their ideas and designs for another brainstorming session. 

Agenda:

Interview 1:

1. General introduction of the team and Mary
2. Thank for Mary for taking the time to do the interview, and ask what encouraged to join the project
3. Briefly explain project and establish the expectations we have
4. Ask what she does in daily life (try to branch off of the reason she joined the project)
5. Ask about hobbies
6. Ask about difficulties she has in her daily life
7. Ask about if she has any ideas for projects
8. Brainstorm some potential solutions to problems that Mary may have mentioned from the last few questions
9. Summarize the meeting
10. Thank Mary for coming and let her know we will keep her updated about our ideas.

Interview 2:

1. Short recap of last meeting, briefly discussing any key takeaways [5-10 Minutes]
2. Discuss any topics/ideas Mary may have thought of since last meeting 
3. Go over our current project ideas [5 minutes per idea, 20 minutes total] 
4. Get feedback from Mary, ideally try to narrow down to 1-2 most appealing ideas
5. Decide whether or not to move forward with one specific project, or to keep brainstorming

Summary and Takeaways:

Through the interviews we learned a lot about Mary’s activities and daily life. We were inspired by her strive to be independent and the many techniques she’s used to do her tasks on her own as much as possible. Although we were happy to learn Mary is very independent, this initially made it difficult to come with solutions that Mary would truly benefit from. It is why our first project idea was a solution to help with difficulties related to physical concerns such as reaching for items at the back of a closet or refrigerator. During the first interview we had discussed solutions to this problem but after a conversation with the professor, we realized our solution was very mechanical. And so, we went back to our notes from the first meeting and held a second interview with Mary. 

During the second interview, we clarified the purpose of the project was to find solutions that involve electronics. We eventually learned that Mary loves to write whenever she gets the chance. We wondered if Mary has difficulty noting ideas that she may randomly think of. As a result we suggested a device that could record notes by the press of a button. This way, Mary doesn’t have to worry about the time and difficulty of typing ideas. Instead, she simply scrolls through her recordings, minimizing hand movement and maximizing her ability to write more stories. 

Images of Mary’s fridge from the first Interview. We wanted to see the layout of her fridge when we were discussing the accessibility problem with items at the far back.

Design inspired from our second interview. This device is a recorder that would help Mary note ideas whenever and wherever.

Thoughts:

The two meetings we had with Mary both went well. Mary is a very positive lady who loves her life so much, which inspires us not only for this assignment but also for our life attitude. We followed the agenda to carry on the meeting and Mary cooperated with us nicely. Unfortunately we could not meet in person for the second interview. But even over the phone, we had a productive discussion about the pros and cons of different project ideas. We appreciated Mary’s honesty and feedback on our ideas as it greatly helped us revise our designs. Since everything went so well, we ended the meeting about 10 minutes earlier than expected. The second meeting was truly productive which made us know better about Mary’s needs and helped us decide on the final design: the voice recorder device. All the team members are very excited about this idea and hope the project goes well and benefits Mary in the future.

Our goal with the project is to create a personally tailored device that makes one aspect of our client’s life better. To better meet this goal, we first needed to know more about the life of our client, Allie. Because of the inconveniences of travel for Allie, we decided to meet online for the interview on Monday, March 21st.

Agenda

We decided to follow a loose schedule for the interview as topics can pivot depending on the conversation. The schedule goes as follows:

Icebreaker (5-6 min)

• Introduce ourselves and talk about one of our hobbies
• Ask Allie about her hobbies
• What drew her to participate/volunteer for this project
• Also, ask Allie if we can record the meeting

Explaining the project (5 min)

• We are trying to build a helpful divide, but we are not professionals. We will try our best, but the final product might not work.
• Give a quick timeline overview of the process; April 6th Prototype Crit; May 2nd Final Crit
• Ask Allie if she has any general questions regarding the project.

Discussing potential ideas (35 min)

• Ask Allie to draw or describe her daily routine
• Allie might have some ideas on what she wants to see built, might leave some time for that
• Then ask her about which tasks she enjoys doing, dislikes doing, or finds challenging to do; something she enjoyed doing and had to give up because of inconveniences
• Ask her to elaborate on the challenging tasks and start coming up with potential ideas as a group
• Mockup solutions with materials at hand or have a whiteboard on zoom so everyone can draw on it

Wrap Up and Conclusion

• Thank Allie for her time
• Final documentation
• Reiterate schedule if needed

Summary + Takeaways

We started the meeting by talking about our hobbies as an icebreaker. We learned that Allie is a very creative and social person. She said that she likes designing posters on the computer, painting, shopping, and going out to eat. When we asked about her daily routine, she told us that she spends most of her day on the computer, designing with canva and taking skillshare classes. As we got deeper into the conversation, Allie told us that she relies a lot on her helper to assist with day-to-day tasks, and she had some ideas for assistive devices that could give her more independence. She said that a grabber that could pick up objects that she drops off her bed at night could be useful, or an automatic wallet or cash box that could hand out the right amount of money to a cashier and receive the change in return when she goes shopping. 

Brainstorming ideas with Allie: Automatic money dispenser for shopping and self-correcting spoon to reduce spilling food

We also chatted about the different sources of inspiration for Allie’s posters. She told us that she creates posters for anything from stores to movies to her favorite foods. In a follow-up email, she showed us some of the posters she has made. Allie seems to draw a lot of her artistic inspiration from the world around her, so we also started to brainstorm ideas for devices that could challenge her to create posters related to different topics or objects.

Babe movie poster (top) and Toys R US store poster (bottom) designed by Allie

Post-Meeting / Team Thoughts

We all enjoyed being able to connect and chat with Allie! Going into the interview, we believed that Allie would have a variety of issues to tackle due to the extent of her disability. However, some of the problems she presented were past our experience level. We quickly realized that it would be tough for us to create devices to retrieve dropped items, pick specific items out of cabinets, or allow her to use a spoon without spilling. It was disheartening to tell her that we may not be capable of solving those types of problems, but Allie was so positive and friendly that it made it easy to work with her. 

After the meeting, we had plenty of ideas to work on, but once we started breaking down these ideas, they all seemed fairly difficult. A money box that could pay for items and receive change seemed like the best idea in terms of practicality and usefulness to Allie. However, we realized it could be difficult to make the money dispenser consistently work. We also came up with an idea to create an animated Tamagotchi-like game that would give her new ideas for poster designs. We thought this device could be cute and fun for Allie, but this didn’t include very many physical computing components. Finally, we brainstormed a design for a self-correcting spoon. This concept was interesting and useful for Allie’s day-to-day life, but the mechanics would be difficult to achieve on a small scale.  We decided to focus on the money box idea moving forward as it seemed to be the most feasible and could be simplified if need be. 

Overall, we all agreed that the interview with Allie went smoothly. We followed our agenda for the most part, but we also didn’t force ourselves to stick to a rigid schedule so that we had room to let the conversation flow naturally. This helped us come up with more ideas and get to know Allie better as a person. If we were to do anything again, we would have asked Allie about some of the other hobbies she mentioned during the icebreaker that we didn’t have time to discuss in-depth. This information could have given us even more ideas for assistive devices to consider.

The SunLight

Cassie Rausch

The SunLight is an alarm clock that relies on a bright light to wake up the user rather than a loud noise.

I chose to keep the controls very simple and almost utilitarian-looking.

A side view of the SunLight showing all of the electronics.

Process

When I was designing the exterior of my alarm clock, I originally wanted to have two separate parts, the light made to look like a sun and the clouds that it would emerge from. The clouds would only block the first half of the suns arc and the sun would be moving from the point that the alarm was set until the end.

Sketches of potential sun alarm ideas.

More sketches of potential sun alarms, closer to what I originally had planned for the project.

I knew when I was designing this project that I wanted the alarm to be a physical movement of the light or a cover or something similar instead of the whole system being digital. The idea of a light blinding you in the morning to wake you up is pretty harsh, so I wanted to make this a more enjoyable and gradual experience for the user.

Sun cover for the light that I designed in Solidworks and 3D printed in PLA plastic.

I was pretty excited for my cute sun cover, but I realized that I would have to drill holes to let light out because I made the cover too thick. The fit was also a bit snug, but I could get the LED in and out with some effort.

The cloud covers sitting out drying after I spray painted them.

Spray painting the cloud covers took a bit of practice before I knew that I could get the gradient down. I didn’t have much experience with spray paint, but I had a lot of fun painting them.

I made two big decisions during my design process that led me to my final result. First was the decision to make the whole shape a cloud and make the gradient from dark to light using paint. When I was modeling my ideas in Solidworks, I realized that it would be difficult to make something with more volume because of the size of my project. 3D printing a cloud like shape would be too large to fit on the print bed and if I make a mistake it’s expensive and time consuming to reprint. Instead I decided to use the laser cutter to save time and made the whole shape a cloud with the electronics and light sandwiched in the middle.

Deciding how I’m going to lay out my design and make the sandwich idea work.

My next big decision in my process was deciding to put a cover on the servo to block the light rather than attaching the light to the servo. When I was working on the mechanism to move the light I really struggled to get the servo to move the small amounts that I needed to get the slow sweep that I wanted. This was because the heat synch on the LED added a ton of weight that the little servo could not handle. So I tried a few different attachment styles for the LED to the servo (even making a four bar mechanism to better support the weight out of finishing nails and wood scraps) but instead chose to make a much lighter cloud to cover the LED that would be attached to the servo.

First attempt to connect the light to the servo using hot glue and foam board scraps.

Second attempt to attach the LED to the servo. I shortened the arm that the servo uses to provide more mechanical advantage and used a stronger material.

Third attempt to connect the servo to the LED. I made an overly-complicated four bar mechanism that I thought would solve the issue.

Discussion

I really enjoyed this project and the creative freedom that it allowed me to make whatever I want. As someone who is constantly thinking of things that I’d like to make or try out, I don’t often have the time or materials to be able to make many of my ideas, so I’m glad that I got to try this one.

This project mainly taught me that I take much more time when considering the code than the mechanical aspects of the project, and I also find the code much less interesting. I remember spending large amounts of time debugging my code and making it function correctly and it was one of the most infuriating experiences. Especially when working with relatively cheap electronic parts and my limited knowledge of electronics, it was difficult to tell at times if the issue was coding, human error, wiring, or the parts. However I do remember every step of building the project very well. I really enjoyed designing and building it, although I do wish that I had used better time management.

Despite the frustrations, I am very happy with how this project turned out. It ended up looking pretty different than I had originally imagined, but it also pushed me into some minimalist designing that I typically wouldn’t do. It doesn’t function perfectly, but I’m proud of how much I learned while coding and how much I was able to do.

Looking at what others thought of my project, I received the comment that “This is super aesthetically pleasing and I think the design of the clouds/sun is really impressive – it’s similar to storytelling with physical computing.” The idea of storytelling with physical computing really interested me, and I liked the way that this person saw my project. I also received the suggestion that “I think having more controls to alter the light intensity during the day would be useful.” I somewhat worked the idea of the SunLight also working as a lamp into my concept, but due to limited time I didn’t work this in. Although I didn’t consider an additional control to change light levels and I think that could really polish off my design.

I really enjoyed this project, and I plan to continue to make changes to my SunLight. My next step is to integrate the power source into the design and soldering the wires, followed by making the transition from light to dark more seamless, then adding in the lamp function. If I could, I’d like to give the lamp a brightness dial as well as make an option to change color. I would also make the dial to set the hour more accurate and easy to read.

Technical Information

Block Diagram

Schematic

Code

//Project 2: SunLamp
//Cassie Rausch
//pin 7 controls the servo
//pin A0 controls the potentiometer
//pin 3 controls the LED
//pin 9 controls the tactile switch
//Debugging help from Michelle Zhu


#include <Servo.h>
#include <Wire.h>


const int servoPIN = 7;
const int potPIN = A0;
const int ledPIN = 3;
const int switPIN = 9;



Servo sun;

void setup() {
  // put your setup code here, to run once:
  pinMode(potPIN, INPUT);
  pinMode(switPIN, INPUT);
  pinMode(servoPIN, OUTPUT);
  pinMode(ledPIN, OUTPUT);

  sun.attach(7);

  Serial.begin(9600);
}

void loop() {
  //  maxTime = 12 * 60 * 60 * 1000; uncomment this if using as intended and not for demo
  int maxTime = 30 * 1000;
  int servoMax = 180;
  int knobMax = 1023;

  int knobSelection = analogRead(potPIN); // potentiometer reading 
  //  int moveTime = 15 * 60 * 1000; uncomment this if using as intended and not for demo
  int moveTime = 15 * 1000;
  int waitTime = maxTime * float(knobSelection / knobMax) - moveTime; // num total runtime ms
  int timeInterval = moveTime / servoMax; // time between each tick movement

  // Actual running

  if (digitalRead(switPIN) == HIGH) { // allowed to begin
    Serial.println("Switch on");
    sun.write(0);
    digitalWrite(ledPIN, HIGH); // turn the light on
    delay (waitTime); // don't do anything

    // start the alarm!
    for (int pos = 0; pos <= servoMax; pos++) {

      if (digitalRead(switPIN) == HIGH) { // 1 = true = keep going!
        delay (timeInterval);
        sun.write(pos);
        Serial.println(pos);
      } else { // 0 = stop the alarm!
        Serial.println("Switch off");
        sun.write(0);
        digitalWrite(ledPIN, LOW);
        return;
      }
    }
  }
}

Photos and Videos

Overall Photo

The candy funnel seen without candy.

The candy funnel seen with candy.

Internal Wiring

Process Images

Above is my initial design idea of the Good Habits Checklist. In the initial design, candy was stored in a compartment and once all 5 switches were flipped, a servo would turn a latch, allowing me to open the compartment. Another thing to note was that the display and switches were on the side of the box, which is different from my final design.

Above shows my first prototype. As you can see, I focused on getting the wiring and display before moving on to the exterior box. I switched the number of tasks from 5 to 4 to fit onto the display effectively but did not change much of the design concept at this point.

Shown above is when I began to fit all of my components into the box. This is where the initial design changed the most. After presenting my prototype, other students gave me the idea to create a dispensing mechanism for the candy. I was still able to use the servo that releases candy from a funnel by turning out of the way for a short period of time. Getting the correct timing to dispense a reasonable amount of candy was difficult. Having this new mechanism also allowed me to put the display, switches and LEDs on top of the box. Because I am tall and tower above most tables even when sitting, this allows me to more easily see the display and flip the switches.

Discussion

Overall, I believe that my final critique went well and that I achieved what I set out to do. It was very rewarding to get positive feedback about my candy dispensing system. One student commented that, “I think the candy dispensing is exciting and looks like it functions really well.” Although creating a candy dispenser was not my idea, as I explained in the Process Images section, I thought my idea of the funnel and servo motor was a clever solution. I do, however, wish it was a little more consistent with the number of skittles that come out. Right now, it usually dispensing around 3 skittles but can dispense 1 to 5. Furthermore, students commented how the skittles are launched out onto the desk and can go very far. One student advised me to, “Add a holder for the treats once they’re dispensed.” This is a great idea because it ensures that skittles do not go onto the floor or get dirty from the table. In the end, I truly believe that this will be a useful thing for me to use every day and develop good habits. I am happy with the design and function of the project, but I still plan to fix all the issues explained above. I plan to go back and add a candy holder, tweak the dispenser so it is more consistent, and put an on/off switch so that the batteries do not run out as fast. Then I can get the most out of using the Good Habits Checklist.

Technical Information

Block Diagram

Schematic Diagram

/*
   The Good Habits Checklist

   Tristan Hineman

   Description:
    This program is designed to display daily tasks and read if LEDs are on or off. When an LED is on, the LCD Display will show that the correlated task is completed. When it reads that all LEDs are on, it displays that all tasks are done and turns a servo motor to release candy once.
   Pin Mapping Table:

   Arduino pin | description
   ------------|-------------
   A0            Servo Motor

   5             LED 1
   6             LED 2
   7             LED 3
   8             LED 4

   SDA           SDA pin, LCD Display
   SCL           SCL pin, LCD Display
*/

//Libraries
#include <LiquidCrystal_I2C.h>
#include <Servo.h>
Servo servoMotor;
//Setup to LCD display
LiquidCrystal_I2C screen(0x27, 20, 4);

//Creates and saves a special character that is a check mark
byte customChar[] = {
  B00000,
  B00000,
  B00001,
  B00011,
  B10110,
  B11100,
  B01000,
  B00000
};

//Pin for LEDs
const int led1Pin = 5;
const int led2Pin = 6;
const int led3Pin = 7;
const int led4Pin = 8;

//Pin for servo motor
const int servoPin = A0;

//Variable to run servo motor only once when all LEDs are on
boolean hasRun = false;

void setup() {

  //Pin setup
  pinMode(servoPin, OUTPUT);
  pinMode(led1Pin, INPUT);
  pinMode(led2Pin, INPUT);
  pinMode(led3Pin, INPUT);
  pinMode(led4Pin, INPUT);

  //LCD screen initialization
  screen.init();
  screen.backlight();
  screen.home();
  Serial.begin(9600);

  //Variable for check mark
  screen.createChar(check, customChar);
}

void loop() {
  
  //Attaching library to servo pin
  servoMotor.attach(servoPin);

  //Variable to set servo position
  int servoPos;

  //Variables to read if LEDs are on
  int led1Read;
  int led2Read;
  int led3Read;
  int led4Read;

  //Read and print if LEDs are on
  led1Read = digitalRead(led1Pin) ;
  Serial.print(led1Read);
  led2Read = digitalRead(led2Pin) ;
  Serial.print(led2Read);
  led3Read = digitalRead(led3Pin) ;
  Serial.print(led3Read);
  led4Read = digitalRead(led4Pin) ;
  Serial.println(led4Pin);


  //Displaying text when all LEDs are on
  if (led1Read == 0 && led2Read == 0 && led3Read == 0 && led4Read == 0) {
    screen.setCursor(0, 0);
    screen.print("ALL TASKS DONE!  ");
    screen.setCursor(0, 1);
    screen.print("ALL TASKS DONE!");
    screen.setCursor(0, 2);
    screen.print("ALL TASKS DONE!");
    screen.setCursor(0, 3);
    screen.print("ALL TASKS DONE!");

    //Using hasRun variable to move the servo motor only once
    if (hasRun == false) {
      servoMotor.write(90);
      delay (100);
      servoMotor.write(180);
      hasRun = true;
    }
  }
  else {
    hasRun= false;
    servoMotor.write(180);
    //Displaying the task and an X when the LED 1 is off
    if (led1Read == 1) {
      screen.setCursor(0, 0);
      screen.print("1)Eat Breakfast:X");
    }

    //Displaying the task and a check mark when the LED 1 is on
    else {
      screen.setCursor(0, 0);
      screen.print("1)Eat Breakfast:");
      screen.write(check);
    }
    
    //Displaying the task and an X when the LED 2 is off
    if (led2Read == 1) {
      screen.setCursor(0, 1);
      screen.print("2)Exersize:X      ");
    }

    //Displaying the task and a check mark when the LED 2 is on
    else {
      screen.setCursor(0, 1);
      screen.print("2)Exersize:      ");
      screen.setCursor(11, 1);
      screen.write(check);
    }
    //Displaying the task and an X when the LED 3 is off
    if (led3Read == 1) {
      screen.setCursor(0, 2);
      screen.print("3)Homework:X     ");
    }

    //Displaying the task and a check mark when the LED 3 is on
    else {
      screen.setCursor(0, 2);
      screen.print("3)Homework:      ");
      screen.setCursor(11, 2);
      screen.write(check);
    }

    //Displaying the task and an X when the LED 4 is off
    if (led4Read == 1) {
      screen.setCursor(0, 3);
      screen.print("4)Brush Teeth:X");
    }

    //Displaying the task and a check mark when the LED 4 is on
    else {
      screen.setCursor(0, 3);
      screen.print("4)Brush Teeth:");
      screen.write(check);
    }
  }
}

Overall photo of the chameleon necklace.

Detail Photos:

TCS34725 RGB sensor at the center of the necklace reads the color of surrounding objects.

Wire-wrapped acrylic beads are placed on top of the color-changing RGB LEDs.

The backside of the necklace pendant includes the Teensy3.2 microcontroller, a slide switch, and both RGB LEDs that illuminate the acrylic beads.

Necklace in Use:

The necklace displaying the color of the red breadboard.

The necklace displaying the color of the green breadboard.

The necklace displaying the color of the blue breadboard.

Video:

The necklace changes color to match the breadboards held in front of the RGB sensor. The switch on the backside of the protoboard is flipped to keep a given color displayed on the necklace beads.

Process:

Decision 1:

The first big decision of this project was to use the Teensy3.2 microcontroller over the Attiny85. Although the small size of the Attiny85 made it ideal for jewelry, it wasn’t compatible with the tcs34725 RGB color sensor library. By switching to Teensy3.2, my necklace was a bit bulkier, but this microcontroller was much easier to code with and compatible with all the libraries I needed for my project.

Decision 2:

The second big decision I made was to change my project concept from chameleon earrings to a chameleon necklace. I realized that the dangling wires connecting the RGB sensor on a hidden wristband/necklace to a pair of earrings might be uncomfortable for the user. It felt more practical to contain all of the hardware on a single necklace. 

Chameleon Earrings “works-like” prototype

I changed my project to a chameleon necklace based on this sketch after the prototype presentations.

Process Images:

Comparing the tcs34275 RGB sensor readings of various colors to their actual RGB values.

While soldering the electronic circuit for the necklace, the left side of the necklace was not lighting up. I had to re-solder the wires connecting the output pins to the LEDs on the left side after I realized they were loose.

For the final step of my project, I created a herringbone wire wrap for the acrylic beads and a wire chain. I learned both techniques from YouTube tutorials.

Discussion:

My original goal for this project was to create a pair of “chameleon” earrings with LEDs that would light up in the same color as an object placed in front of an RGB sensor. I created a “works-like” prototype of the earrings for the prototype presentation. A common concern in the feedback I got from the class was that the earrings would have a lot of dangling wires, which might make them hard to wear and very fragile. One classmate suggested that I “try wireless if want to use for earring.” While I originally wanted to have a wireless Bluetooth connection between the earrings and microcontroller/RGB sensor set-up, Zach warned me that the Bluetooth chips we had were probably too big to be used for earrings. I got another comment that “if it is hanging on your neck, maybe you could even turn that into a necklace.” I was initially hesitant to go down this route since I wanted to hide the electronics and feature the color-changing LEDs in the jewelry. However, after doing more sketches of a revised design, I realized that the protoboard and RGB sensor could also look nice as part of the necklace. Moving forward, I changed my project to be solely a necklace instead of earrings with the electronics on a hidden necklace or wristband. 

While working on the chameleon necklace, I experienced the difficulties of creating smaller-scale electronic devices, especially wearables. I wanted to use the smallest possible microcontroller and fit all the electronics onto the smallest possible protoboard to reduce the bulk of the necklace pendant. I started with the Attiny85 and spent almost a week searching for ways to make the Tcs34725 RGB sensor library compatible with this microcontroller. After running into many dead ends, I switched to the Teensy3.2 microcontroller. In hindsight, I wish I didn’t get so hung up on using the Attiny85. If I had moved on to Teensy3.2 earlier, I would have had more time to assemble my necklace at the end, which was another challenging part of this project. While constructing the final necklace, I had to extensively plan how to fit all of the wires, resistors, sensors, and actuators on the small protoboard. Although it was frustrating to resolder the many loose/broken wire connections I found, this process helped me strengthen my soldering skills. 

Overall, I feel like I achieved most of my goals for this project. My necklace can display most rainbow colors and has an eye-catching design. I still think my soldering could be improved as the final necklace still felt fragile, leading to a broken wire on the final critique day.

In the future, I would try to recreate this necklace using the more flexible stranded wires rather than the solid wires, which are prone to breaking. Also, I would like to have four LEDs on the necklace to draw more attention to the color-changing beads rather than the RGB sensor/microcontroller component. 

Block Diagram:

Electrical Schematic:

Code:

/* 60-223 Chameleon Necklace
    Lynn Rushkin

    Description: This code takes the RGB reading from the TCS34275 RGB sensor and sends this RGB value
                 to two RGB LEDs to output light of that color. When the slide switch is in the "on"
                 position, the TCS34275 sensor will take new color readings. When the switch is "off",
                 the sensor will stop taking color readings and the RGB LEDs will continue to display
                 the last color reading the sensor took.

    Pin mapping:
    Arduino pin | type   | description
    ------------|--------|-------------
    10            output    RGB LED pin for color output
    17            input     slide switch pin that reads on/off position of the switch
    20            output    RGB LED pin for color output

    The following sources were consulted to create this code:
    https://courses.ideate.cmu.edu/60-223/s2022/tutorials/button-and-switch
    https://learn.adafruit.com/adafruit-color-sensors/library-reference
    Example code "simple" from the Adafruit_NeoPixel library
    Example code "tcs34725" from the Adafruit_TCS34725 library
*/
//libraries used
#include <Wire.h>
#include "Adafruit_TCS34725.h"
#include <Adafruit_NeoPixel.h>

//pin names
#define PIN1        20
#define PIN2        10
const int switchPin = 17;

//other variables
float red, green, blue;

//defs for RBG LED
#ifdef __AVR__
#include <avr/power.h> // Required for 16 MHz Adafruit Trinket
#endif

// I used a common Anode RGB LED
#define commonAnode true

//define number of RGB LEDs per strand
#define NUMPIXELS 1 // LED strand length of 1 (both RGB LEDs I used were connected to their own output pins)

//name the LED strips "pixels1" and "pixels2" , both containing only 1 RGB LED at PIN1 and PIN2, respectively.
Adafruit_NeoPixel pixels1(NUMPIXELS, PIN1, NEO_GRB + NEO_KHZ800);
Adafruit_NeoPixel pixels2(NUMPIXELS, PIN2, NEO_GRB + NEO_KHZ800);

//define RGB LEDs brightness:
#define BRIGHTNESS 50

// Time (in milliseconds) to pause between pixels
#define DELAYVAL 250

// our RGB -> eye-recognized gamma color
byte gammatable[256];
Adafruit_TCS34725 tcs = Adafruit_TCS34725(TCS34725_INTEGRATIONTIME_50MS, TCS34725_GAIN_4X);


void setup() {
  pinMode(switchPin, INPUT);
  tcs.begin();
  if (tcs.begin()) {
    Serial.println("Found sensor");
  } else {
    Serial.println("No TCS34725 found ... check your connections");
    while (1); // halt!
  }

  // Thanks PhilB for the following gamma table. 
  // It helps convert RGB colors to what humans see
  for (int i = 0; i < 256; i++) {
    float x = i;
    x /= 255;
    x = pow(x, .5);
    x *= 255;

    if (commonAnode) {
      gammatable[i] = 255 - x;
    }
    else {
      gammatable[i] = x;
    }
  }
  pixels1.begin(); // INITIALIZE NeoPixel strip object
  pixels1.setBrightness(BRIGHTNESS);

  pixels2.begin(); // INITIALIZE NeoPixel strip object
  pixels2.setBrightness(BRIGHTNESS);
}


void loop() {
  int switchVal = digitalRead(switchPin);
  
  //The following if-statement ensures that new RGB readings are taken from
  //the sensor only when the switch is on. 
  if (switchVal == 1) {
    tcs.setInterrupt(false);  // turn on LED

    delay(60);  // takes 50ms to read

    tcs.getRGB(&red, &green, &blue);

    tcs.setInterrupt(true);  // turn off LED
  }
  else {
    tcs.setInterrupt(true);
  }

  //print the following lines of code to check the RGB sensor readings
  Serial.print("R:\t"); Serial.print(int(red));
  Serial.print("\tG:\t"); Serial.print(int(green));
  Serial.print("\tB:\t"); Serial.print(int(blue));

  Serial.print("\n");

  /*After collecting RGB readings from the tcs34275 sensor with all the rainbow colors,
    I found that the readings never dropped below 20 or got higher than 200. 
    To acheive a closer color match in the LED output to the object color input, I mapped the 
    sensor readings from the range of 20 - 200 to the range 0 - 255 for modified color values. 
    */
  int redMod = map(int(red), 20, 200, 0, 255);
  int greenMod = map(int(green), 20, 200, 0, 255);
  int blueMod = map(int(blue), 20, 200, 0, 255);

  // Send the updated pixel colors to the RGB LEDs.
  pixels1.setPixelColor( 0, pixels1.Color(int(redMod), int(greenMod), int(blueMod)) );
  pixels2.setPixelColor( 0, pixels2.Color(int(redMod), int(greenMod), int(blueMod)) );

  pixels1.show();   
  pixels2.show();


  delay(DELAYVAL); // Pause before next pass through loop
}

Tristan’s Final

Nicole’s Final

Close up Image of Infrared Proximity Sensor

Close up Image of DC Motor and Motor Driver

Description

This Double Transducer takes in the brightness of light as an input and converts that into a rotational position and then finally converts that rotational position into a rotational speed. A photoresistor detects the brightness of light hitting it which is then converted to a rotational position between 0 and 180 degrees. The servo motor rotates a popsicle stick to this position and an IR sensor detects how far away the popsicle stick is. The IR sensor sends this information back to the Arduino and the Arduino maps this value to an amount of voltage. This voltage is then sent to a DC motor, the transducer’s final output. Based on the voltage supplied the rotational speed of the DC motor varies between 20 and 60 rpm. 

Progress Pictures

Figuring out the distance between the IR sensor and popsicle stick.

Having trouble with pulley system.

We tested many different layouts for the components on the board to determine which was the most sensible.

Testing the photoresistor range of inputs to determine the range to use in the map function

Discussion

Our final project was a successful double transducer but there were many unforeseen challenges and modifications we had to make to our original idea. The preliminary middle step was to have a servo motor rotate a pulley, moving an index card back and forth. In theory this seemed like a simple mechanical system to construct, however all of our available options for a rope did not work. There was too much friction between the rope and the 3-D printed part for the pulley to successfully rotate the rope. Unfortunately, we had to scrap the pulley system idea and come up with another way for the servo motor to move the popsicle stick to various positions. At this point we had already wired the IR sensor and written the code for it so we didn’t want to rethink the entire middle step. We decided to instead attach the popsicle stick directly to the servo motor and then position the IR sensor to detect how far away it is. This ended up being much easier to execute than our original idea and was also a more reliable system. Being forced to deviate from our original idea was unsettling at first but worked out for the best in the end. Having to pivot taught us that you will often have to rethink and rework your original plan and to be prepared for failure when you attempt to execute your ideas. The importance of having an adaptable mindset is just as important as the planning process when working on physical computing projects. 

Another misstep we faced was due to soldering. I did not truly understand the importance of precision when soldering until an overlooked error led to me short circuiting my Arduino. At first I did not understand why attaching the board I had soldered to my Arduino resulted in my Arduino immediately shutting down. My first instinct was that it was a faulty wire, but after testing all my wires and all of them working I knew it had to be from the way I soldered. When I looked at my soldering more closely I realized that there was melted metal connecting ground and power which was causing the entire system to fail. Surely enough when I fixed this error the board and the Arduino started working again. This misstep taught me it is important to not rush soldering and if a system is failing to check if it’s from a soldering error.  

Writing the code was straightforward and didn’t present many challenges. However, determining the ranges to use in the map function was more difficult than anticipated and required more trial and error than expected. A lot of times our transducer not working was attributed to inaccurate ranges. At first we struggled to identify the optimal range for the transducer to work. As we continued to aimlessly plug and chug different numbers we realized that by interacting with the device we would get a better understanding of what values are needed for the map function. Another challenge was that the values of the inputs in the system, the photoresistor and IR sensor were often changing drastically. We learned that the IR sensor’s values were changing due to the potentiometer moving so we made sure it was more secured in one position. The photoresitor’s values were changing due to a change of environment, however as long as the transducer was in the classroom the same range of values worked. Adjusting the ranges taught us how to be more systematic about how we test values for inputs and outputs for the future. 

Prior to this project, we both had limited experience with wiring inputs and outputs to the Arduino. From working on the double transducer we learned how to wire and program a photoresistor, IR sensor, LCD display, servo motor, and DC motor. This experience made both of us more comfortable with the challenges that come along with using these various inputs and outputs. For this project particularly, the DC motor presented the most frustration. Unlike the servo motor that can be wired directly to the breadboard, the DC motor needed an additional power source and assistance from a motor driver in order to work. We spent a while toying around with the DC motor before learning that its failure was due to the absence of a motor driver. This set back taught us to not expect similar inputs and outputs to work the same. The DC motor and servo motor may function similarly but their wiring and software are different. This project was a valuable introduction to physical computing and allowed both of us to become familiar with using several new inputs and outputs.

Functional Block Diagram

Schematic

Code

/*
   Double Transducer: Light Brightness -> Rotational Speed

   Team Members: Nicole Monaco, Tristan Hineman

   Description:
    The program is disigned to recieve input from the photoresistor and covert
    the value to an angle for the servo motor with a popsicle stick to move to
    where then an infrared proximity sensor will detect the distance of the popsicle stick and convert that to
    a rotational speed between 20 and 60rpm for the DC motor to output.

   Pin Mapping Table:

   Arduino pin  |  description
   -------------|-------------
   A0             Photoresistor

   A2             Servo Motor
   A1             Infrared Proximity Sensor

   5              Driver chip for DC Motor

   SDA            SDA pin, LCD Display
   SCL            SCL pin, LCD Display


   References:
   https://courses.ideate.cmu.edu/60-223/s2022/tutorials/reading-a-photoresistor
   https://courses.ideate.cmu.edu/60-223/s2022/tutorials/servo
   https://courses.ideate.cmu.edu/60-223/s2022/tutorials/IR-proximity-sensor
   https://courses.ideate.cmu.edu/60-223/s2022/tutorials/I2C-lcd
*/

//Libraries
#include <LiquidCrystal_I2C.h>
#include <Servo.h>
Servo servoMotor;

//Pin for photoresistor
const int photoPIN = A0;

//Pin for infrared proximity sensor
const int irPIN = A1;

//Pin for servo motor
const int servoPIN = A2;

//Pin for dc motor driver chip
const int dcPIN = 5;

//Variables for LCD display
unsigned long lastServoTime = 0;
unsigned long lastIRTime = 0;
unsigned long lastDCTime = 0;
unsigned long lastLCDTime = 0;

//Setup to LCD display
LiquidCrystal_I2C screen(0x27, 16, 2); // create LCD display object
void updateScreen(int photoVal, int servoPos, int IRreadVal, int DCSpeed) {
  screen.setCursor(3, 0);
  screen.print(map(photoVal, 80, 300, 0, 90)); 
  screen.setCursor(11, 0);
  screen.print(servoPos);
  screen.setCursor(3, 1);
  screen.print(map(IRreadVal, 45, 960, 0, 90)); 
  screen.print(map(DCSpeed, 50, 80, 36, 60)); 
}

void setup() {

  //Pin setup
  pinMode(photoPIN, INPUT);
  pinMode(irPIN, INPUT);
  pinMode(dcPIN, OUTPUT);
  servoMotor.attach(servoPIN);

  Serial.begin(9600);

  //LCD screen initialization
  screen.init();
  screen.backlight();
  screen.home();
  screen.print("i:");
  screen.setCursor(7, 0);
  screen.print("m1:");
  screen.setCursor(0, 1);
  screen.print("m2:");
  screen.setCursor(7, 1);
  screen.print("o:");
}

void loop() {

  screen.setCursor(5, 0);
  screen.print (" ");

  //Variable to store photoresistor value
  int photoVal = analogRead(photoPIN);
  Serial.print(photoVal);
  Serial.print("     ");

  //Variable to set servo position
  int servoPos;

  //Adjust the angle of the servo motor according to the registered light brightness from the photoresistor
  if (millis() - lastServoTime >= 500) {
    servoPos = map(photoVal, 80, 300, 0, 45);
    servoMotor.write(servoPos);
    lastServoTime = millis();
  }

  //Variable to store infrared proximity sensor value
  int IRreadVal;

  //Read and print infrared proximity sensor value
  IRreadVal = analogRead(irPIN);
  if (millis() - lastIRTime >= 50) {
    lastIRTime = millis();
  }
  Serial.println(IRreadVal);

  //Variable to set the speed of the DC motor
  int DCSpeed;

  //Adjust the speed of the dc motor accoding to the distance from the infrared proximity sensor
  if (millis() - lastDCTime >= 500) {
    DCSpeed = map(IRreadVal, 45, 960, 50, 80);
    analogWrite(dcPIN, DCSpeed);
    lastDCTime = millis();
  }

  //display on the LCD
  if (millis() - lastLCDTime >= 500) {
    updateScreen(photoVal, servoPos, IRreadVal, DCSpeed);
    lastLCDTime = millis();
  }
}

Overall photo of Lynn’s double transducer

Overall photo of Yongwen’s double transducer

Detail Photos:

The lever switch connects to the first breadboard (the knight is trying to save the princess by tapping the switch)

Wiring details

The photoresistor detects the LED brightness

The second breadboard assembles the last wires to connect to the servo motor

The arm is connected to the servo motor which can lift the platform up from 1 inch to 4 inches (the princess is being attacked by the dragon, please save her!)

Videos:

Video demonstration of Lynn’s double transducer.

Video demonstration of Yongwen’s double transducer.

Narrative Description:

Our double transducer takes the tapping rate on the lever of the lever microswitch as an input. The speed of tapping on the lever translates to the brightness of the LED. The LED brightness is read by the photoresistor and dictates the position of a servo motor arm which lifts a princess cut-out up and down. An LCD screen will read out the tapping rate, LED brightness, photoresistor reading, and hobby servo motor arm position. 

Discussion:

Overall, this project presented a valuable opportunity for us to become more familiar with coding in C and learn the electrical workings of different sensors and actuators. Both of us came in with different skill sets. Yongwen provided many creative ideas, such as turning our double transducer into a game of sorts where a princess was tied to the lifting arm of the servo motor and trying to escape a dragon cut-out at the bottom of the platform. She also provided insight into the most visually appealing layout of our final project. Lynn came in with some coding experience in python, so she was able to take a larger role in debugging our code when we were not getting the LCD readings or outputs we expected. By the time we completed the final product, we had both picked up on many new skills from each other.

Although we ran into challenges with almost every step of this project, this allowed us to learn a lot about the wiring and coding required to operate complex electrical circuits and devices. Working with the lever microswitch was one of the most challenging parts. One of the first decisions we had to make was how to perform our tapping rate calculations. Initially, we tried tracking the tapping rate as the number of taps each second but soon realized that we wanted the tapping rate to update faster than each second. Moving forward, we chose to track the tapping rate by measuring the time between taps so that the rate could update after each tap. We also ran into issues of our code outputting higher tapping rate readings than expected. After debugging, we realized that our code was considering each time the switch was in the “on” position as a tap. This meant that if the switch was held down for longer, a single long tap was read as multiple tapping events. We solved this issue by adding an “if statement” to only count a tap when there was a transition from the “off” to the “on” position. Even after fixing this in our code, we still were getting tapping rate values that were too high. After talking to Professor Zacharias, we learned that our switch could potentially be experiencing input “bouncing,” which is essentially electrical noise in the device that causes it to bounce between on/off states when tapped. We were able to “debounce” our input with a simple delay of 3 milliseconds after each switch reading to avoid reading the electrical noise. By taking the time to understand the intricacies of how the lever micro switch operates, we successfully created code that gave an accurate tapping rate reading. 

If we had more time with this project, we would have chosen a more complex middle step over an LED and photoresistor. Since we were confident that we could get the LED output and photoresistor input working within the time frame of this project, we decided to go in that direction and focus our efforts on developing a code to get an accurate tapping rate reading for the lever microswitch and achieving a vertical lift with the angled servo motor arm. Now that we are more experienced in coding, soldering, and wiring, we could experiment with new sensors and actuators such as a magnetometer for magnetic field strength, an RGB sensor for color, or a buzzer to emit different sound pitches.

Progress Photos:

Lynn trying to find the best way to count the tapping rate for the input.

Measuring the correct distance for servo motor arm height of 1 inch to 4 inches.

Our first attempt at soldering the LED and photoresistor to a breadboard.

Beginning to assemble our double transducer with the LCD display.

Block Diagram:

Electrical Schematic:

Code:

/* 60-223 Double Transducer
    Lynn Rushkin and Yongwen Dai

    Description: This code takes the tapping rate read by a lever microswitch and maps that to
                 the brightness of an LED. The level of light emitted from the LED is read by a
                 photoresistor which maps that value to the angle of a hobby servo motor arm.

    Pin mapping:
    Arduino pin | type   | description
    ------------|--------|-------------
    A2            input     Photoresistor that senses LED brightness
    6             input     Micro lever switch that reads on/off position of the switch
    3             output    LED pin for brightness
    7             output    hobby servo motor for arm movement

    The following sources were consulted to create our code:
    https://courses.ideate.cmu.edu/60-223/s2022/tutorials/servo
    https://courses.ideate.cmu.edu/60-223/s2022/tutorials/I2C-lcd
    https://courses.ideate.cmu.edu/60-223/s2022/tutorials/debouncing
    https://courses.ideate.cmu.edu/60-223/s2022/tutorials/button-and-switch
    
Double Transducer: Frequency of Tapping to Distance to Beep Pitch

*/

#include <Wire.h>
#include <LiquidCrystal_I2C.h>
#include <Servo.h>
Servo liftingArm;
LiquidCrystal_I2C screen(0x3F, 16, 2);

//pin names
const int LEVERPIN = 6;
const int LEDPIN = 3;
const int PHOTOPIN = A2;
const int ARMPIN = 7;

//other variables
unsigned long taps = 0;
unsigned long lastSecond = 0;
unsigned long lastThirdSecond = 0;
float tapRate = 0;
int switchVal = 0;
int lightVal = 0;
int liftingVal = 0;
int photoVal = 0;
int currSwitchVal = 0;
int prevSwitchVal = 0;
unsigned long tapB = 0;
unsigned long tapA = 0;

//signal ranges
const int light_low = 0;
const int light_high = 255;
const int photo_low = 5;
const int photo_high = 630;
const int lift_low = 3;
const int lift_high = 75;

//Initialize the time between LCD display updates, and the timer itself.
const int LCDUPDATETIME = 250;
long prevDisplayTime = 0;


void setup() {
  pinMode(LEVERPIN, INPUT);
  pinMode(LEDPIN, OUTPUT);
  pinMode(PHOTOPIN, INPUT);
  liftingArm.attach(ARMPIN);
  Serial.begin(9600);
  //LCD Screen setup
  screen.init();
  screen.backlight();
}

void loop() {

  //Check current time.
  long currentTime = millis();
  currSwitchVal = digitalRead(6);

  /*when there is a transition in the switch position from off to on, count a tap
     tapB tracks the time of the most recent tap, and tapA tracks the time of the prior tap
  */
  if (currSwitchVal == 1 and prevSwitchVal == 0) {
    delay(3); //debounce input signal
    tapB = millis();
    tapRate = 1000.0 / (tapB - tapA);
    if (tapA == 0) {
      tapRate = 0;
    }
    tapA = tapB;
    //set 3 taps/sec as the maximum tapping rate
    if (tapRate > 3) {
      tapRate = 3;
    }
  }

  //set tapping rate to 0 if over a second has passed since the last tap
  if (millis () - tapA > 1000) {
    tapRate = 0;
  }
  prevSwitchVal = currSwitchVal;

  //update the LED brightness and servo motor arm height every second based on the tapping rate
  if (millis() - lastSecond >= 100) {
    lightVal = map(tapRate * 10, 0, 3 * 10, light_low, light_high); //map tapping rate to a brightness value
    analogWrite(LEDPIN, lightVal); //tell LED how bright it should be
    photoVal = analogRead(PHOTOPIN); //photoresistor sends brightness value to computer
    liftingVal = map(photoVal, photo_low, photo_high, lift_low, lift_high); //map brightness value to angle of arm
    liftingArm.write(liftingVal); //tells servo motor arm how high to lift
    delay(10);
    lastSecond = millis();
  }

  //Update LCD display every 250 milliseconds.
  if (currentTime - prevDisplayTime >= LCDUPDATETIME) {
    screen.clear();

    //Map tap rate to a range of 0-99 on LCD display.
    int newtapRate = map(tapRate * 10, 0, 3 * 10, 0, 99);
    screen.home();
    screen.print("i:");
    screen.print(newtapRate);

    //Map brightness value to a range of 0-99 on LCD display.
    int newlightVal = map(lightVal, light_low, light_high, 0, 99);
    screen.setCursor(6, 0);
    screen.print("m:");
    screen.print(newlightVal);

    //Map photoresistor values to a range of 0-99 on LCD display.
    int newphotoVal = map(photoVal, photo_low, photo_high, 0, 99);
    screen.setCursor(8, 1);
    screen.print(newphotoVal);

    //Map height values to a range of 0-99 on LCD display.
    int newliftingVal = map(liftingVal, lift_low, lift_high, 0, 99);
    screen.setCursor(12, 1);
    screen.print("o:");
    screen.print(newliftingVal);

    prevDisplayTime = currentTime;

  }
}