Introduction

For the prototype, we started working on the memory aid that Jim identified would be a great asset in his life. Essentially, it would serve as a reminder as he’s rushing out the door to make sure that he has everything that he needs for his various meetings/that day. If you would like to read about how the initial meeting went where we identified this need, feel free to check out our documentation below.

Meeting Documentation

Product

Overall view of prototype

TFT Display and Motion Sensor

TFT Display connected to ESP32 (goal)

Our prototype demonstrates similar behavior to the final result we are trying to achieve as far as the hardware component.  The motion sensor detects movement so that the system can save energy. The touchscreen display features a button that allows the user to toggle between buttons under a list of everyday items for Jim. Changing the red button green means that the user (Jim) has answered yes to having all the items he needs and is ready to leave his house. The current prototype works with inputs (the items) dictated by the user but ultimately we would like to have the display connected to the ESP32 (image above) and for the ESP32 to communicate with Jim’s Google Calendar so that it can easily know what he’ll need for every day.  Unfortunately we didn’t receive the ESPs until Monday so we had to temporarily set everything up with an Arduino for this demo. The final product is intended to be stored inside a box next to Jim’s front door that can be portable.

Process

Whiteboard diagram back when the idea was to host a server using a raspberry pi instead of Google Calendar

Drawing out the display sizes we could find online with a marker and ruler to debate each option, taking into account price and later shipping times

Testing if the motion detector was working as well as the range by giving it a time out in the cubbies

Sketches of icons for the common items that Jim often needs

First attempt to connect tft display to esp32

Jumpers soldered for SPI use

Record of pins between display and esp32

Reminder screen that displays once motion detector is triggered with daily items listed out (day of the demo)

 

Process

One of the major challenges for us was connecting all the ESP pins to the TFT Display pins. One of our team members is currently working on tackling this issue but we didn’t have time to do it before the demo. The bulk of the problem is the ESP32 having way more pins than the Arduino and figuring out how those pins act based off the schematic included. The majority of the other pieces (calibrating the motion sensor, connecting the motion sensor to the arduino, researching APIs to incorporate Google Calendar) was fairly easy to troubleshoot except for the API data which requires more research to implement.

We received some great advice/ideas/suggestions from this crit. During the in class portion, we received a suggestion that we should assign items to recurring events so Jim doesn’t have to manually log the items in every time. As soon as we got time to talk to Jim one on three, we asked him to send us a list of common items that he needs whenever it’s convenient for him so we can specifically look in Google Calendar for the event name and assign the items as a result. He would still have to enter the items manually for new events, however. Zach came around and gave a critique that Jim may want to use his descriptions and it would be better if the code could parse for a keyword such as list: to start the list.  He also told us that he often abbreviates common events – such as GOM for Gathering of Men which will be useful for us to keep an eye out for when the Google Calender API is up and running.

We received some general aesthetic feedback as well that bigger text and a bigger screen if possible would be nice. He also told us that he would prefer to confirm he has all his objects at the end, rather than individually checking items as he goes. We also suggested there could be some sound from the box or a phone notification in case he bypasses the box without clicking cancel (something we have to implement for the final product). Jim gave us the helpful feedback that though his phone is often on him, he doesn’t always have it on so a phone notification might not be the best method. We also got into a handy conversation with Zach about using a 9V battery vs plugging the device into the wall. As a result, we now know to keep an eye out for the distance between the front door and the nearest outlet.

Our next steps for this project is to get the Google Calendar API up and running so that we can make sure everything is working right/design the user experience from the web portion of the project.  We also have to resolve how to get the TFT Display Screens to properly communicate with the ESP32. Lastly, we must figure out how to display an icon image on the display screen or draw the item out by coordinates for better legibility and to make it more interesting.