INTRODUCTION

For this project, our group of three were tasked with creating a device that would help make the life of our older friend, Jim (as he prefers to go by), better. Though we’re limited by what we’ve learned so far, we were excited to face this task on and to help make something that would survive past the end of this class that would benefit someone else. From our initial first meeting, it is clear to see that the device that would benefit Jim the most would be a device to make sure he doesn’t forget what he need when he leaves the house. We also have another post highlighting what occurred during the first meeting here as well as this post of where our prototype was at about three weeks ago.

WHAT WE BUILT

Description

Our project is a box that is to be installed by Jim’s front door that contains a screen that will display Jim’s items of the day and a motion sensor that triggers the device when he approaches the door. The Google API parses through Jim’s Google Calendar to identify a list of common events or take a list of items from the description of non-common events to display in a list format on his door. Whenever he walks approximately 3 meters by the door, the motion sensor will trip, turning on the display that will remind him to make sure he has all the items he needs for the day.

Final Photographs

Overall Content Photo

Standard opening screen

Barrel jack power and ground connection at the base of the box with motion sensor clearly visible

Temporary velcro closing mechanism

Items list screen(1/3)

Keyboard screen (2/3)

Goodbye screen (1/3)

 

Narrated Sketch of the Intended Use

Every morning, Jim wakes up to a busy day filled with community activities and long board meetings. Those days often require Jim to bring with him many important supplies, without which his day is not as productive. Often, Jim makes note of the things he needs in his Google Calendar, but that is often in a place that is not as easily accessible to him when he is rushing out the door. Imagine this scenario: Jim is walking out the door in the morning, hoping he remembers everything he needs for the day. Just then, a motion detector by the door detects his presence, and, in response, an LED touchscreen lights up, asking Jim if he has everything he needs for the day based on the entries in his calendar. If the answer is yes, then Jim touches the on-screen button to indicate as such, and off he goes, day’s supplies in tow. This is Jim’s daily reminder box helping make his life easier and more productive, one notification at a time.

HOW WE GOT HERE

Our original idea was to have the ESP connect to a server that parses data from Google API to interpret Jim’s calendar for the day. Then the items for those events would be communicated back to the ESP to be displayed in the form of a checklist on the TFT Display as a reminder when he leaves the house.

Sketch of one of the earliest planning diagrams when the Arduino was incorporated

Decided not to set up server. We ended up getting recommended to avoid setting up a server, however, because it would have been really difficult and cost money to maintain. Instead we redesigned the system to parse all data locally and on-premise, which had the trade-off of taking up significantly more memory on the ESP32, out IoT chip.

Ordering a second TFT Display. The next part of the process was to adapt theĀ Adafruit ILI9341 library to do exactly what we needed on the touchscreen display. After the prototype we ordered a slightly bigger screen based off the request of Jim. This served to be useful in the long run since it allowed two people to work n the screens simultaneously and independently. Catherine’s biggest problem was getting the touch to respond properly either due to wearing down the screen or something wrong with the smaller screen.

Attempt#1 for coordinates and basic code with display

While she tinkered with the bigger screen, Lexi experimented with displaying icons to improve visibility of Jim’s items.

Updating the screen based on input from the Serial Monitor

Initially and for demo purposes, we set up the touchscreen to behave like the Google API by pulling data from the Serial monitor. When the screen would sense something written into the Serial monitor, it would perform a Serial.read() to pull that data and print it on the touchscreen.

Serial and display communicating

Getting a keyboard on the display through the ESP. We discovered that there were many problems implementing touch onto the TFT display when connected to the ESP.

Tft display with touchscreen keyboard

We needed to have the keyboard so that Jim could input the verification required to access his wifi once he moved houses. This is because the ESP will not be able to communicate with anything else in the system until it is connected to the WiFi. Ultimately we decided that it’d be better to prioritize the touchscreen keyboard and connected the display to the Arduino which would in turn communicate with the ESP through the RX and TX pins. This also resulted in the scrapping of the icons due the example pictureEmbeded only working with an ESP.

Icon conversion issues. The first problem with the icon displaying was that it took Lexi a week to realize the example code, Adafruit ILI9341’s pictureEmbeded, wasn’t designed to work on Arduino Unos, which she had been testing with. This code doesn’t work with an Arduino, however, so we had to scrap it. Even before we scrapped it though, we were running into the issue that the image converter Lexi was using to the transfer the pixel measurements of the icon into hex codes kept returning values of at least 15,500 values. uint16_t is able to store at most 14,000 values and experimenting with image sizing and various settings would either increase the number of values or have it stay the same. We ended up ditching the entire idea before she had to attempt to find another image converting method.

Image converted into hex codes

Router to connect ESP to Wifi. It would have been useful to discover earlier that a router could be set up to bypass the issue of the ESP connecting to CMU-SECURE. This is due to the way the ESP handles the wifi verification. Since it is much harder to connect to WEP-Enterprise networks, testing anything on campus was impossible. It would have been nice to realize that a router was an option a few days before we did. At that point, there had already been a few days of Catherine testing various bits of code that Sana wrote and implementing at Catherine’s house due to Sana using the school wifi at home.

Memory Problems with the ESP. The ESP32, despite being a chip that can access the internet, holds a very small amount of memory. Accessing the Google Calendar API returned an extremely long string of characters, which was large enough to cause overflow in memory. As a result, the ESP32 turned out being the wrong device for us to use for this project.

Soldering all the wires together to make the connections inside the box more durable

Testing the power and ground side of the DC Jack using the multimeter

Velcro-ing the box closed. Originally we were thinking about gluing the box closed with everything inside it.

The box post-soldering pre-velcroing

Due to the impermanent nature of our box at the time of the demo, we decided to velcro the box closed. This was because velcro is the easiest way of closing the box without prior planning and worked pretty well for what we needed. We realized by doing this that it could actually benefit the box for it to have the option of opening in case anything goes wrong/something gets jostled when Jim moves between houses. As a result, we decided to add latches and a hinge to attach to the final one but didn’t manage to get them in time for documentation.

Velcro-sealed box

 

CONCLUSIONS

The final crit proved to be a very helpful time for us to learn about good features to add to the project. Overall response to the box itself was very positive- both from older friends and from peers. Overall, it seemed as if both groups were very willing to use the box in daily life. Some feedback was that it may have been better to change the design of the box so that it draws a little more attention to itself- like a coat of varnish and maybe a blinking LED light and some sound (these were all suggestions that came from 3 separate people). While we did what our client wanted (these were all ideas we discussed with him), they are certainly ideas we would have implemented.

Additionally, the idea of the demo in its simple form- without the Google calendar implementation- also garnered a good amount of support. It appeared to be a good solution for those who didn’t use a google calendar, though of course there was also the opposing side of the same argument- that the keyboard was too small and the Google Calendar would have worked much better. This reasoning, however, often came from the general sentiment that the screen itself was too small. We’d already bought the largest, reasonably priced screen that we could find, and going any bigger would have given us marginal improvements of size with dramatic price increases. By the end of this discussion, there was a mention that this product would have been great on an iPad, which then begged the question- why not mount an iPad by the door instead? Why this project? To us, this is because an iPad is a one-size-fits-all, expensive solution, whereas our box is a this-size-fits-Jim, more cost-effective solution.

Overall, this project is a quite simple idea, but, under the hood, the complexities of this project were so great that factors like time, WiFi access, and constraints in memory played a big role in making this project much harder to realize. While we were able to put together a final product that satisfied the needs of our client, we would have been much happier with a product that would have been able to access WiFi, a circumstance we couldn’t control. Testing and overall product development were severely impedimented by this, despite the fact that most of the code was written and only needed to be tested. Everything is currently in progress and will soon be updated to meet our standards of functionality, as well as Jim’s. Looking forward, one thing to change would be to ensure that the WiFi chip we are using has much more memory and is able to effectively access the internet from where it is being tested, and to pre-check that the devices being used are always compatible. Compatibility played a large role in the effectiveness of this project. Because IoT projects require hardware, connectivity, and software to work together seamlessly, the presence of many roadblocks in this path made a final product much more difficult to develop with the resources on hand. Still, the learning curve, though steep, was important and taught us a lot about the importance of developing products that are viable and useful. With a larger timeframe, however, we are fully confident that we can produce a fully functioning reminder box for Jim by the time spring semester rolls around.

TECHNICAL DETAILS

Code

The following code is a completely locally-hosted, on-premise code block operating only with the Arduino Uno R3, which takes entries based on the next day to add to the calendar:

/* this code is written for the Arduino component of the reminder box, and is the demo code, 
 * which allows the user to manually enter information about the things he/she needs for the 
 * next day. The reminder box pulls information on the day of the week (set based on an example 
 * schedule here) and prompts the user (whenever the motion sensor is tripped) to check off 
 * whether they have everything or if they want to add anything to the list of things they need
 * for tomorrow.
 */

//libraries
#include <Adafruit_GFX.h>
#include <SPI.h>
#include <Wire.h>
#include <Adafruit_ILI9341.h>
#include <TouchScreen.h>
#include <stdint.h>
#include <DS3231.h>

//Touchscreen pins
#define YP A2
#define XM A3
#define YM 8
#define XP 9

//Calibration data
#define TS_MINX 150
#define TS_MINY 120
#define TS_MAXX 920
#define TS_MAXY 940

#define MINPRESSURE 10
#define MAXPRESSURE 1000

#define IsWithin(x, a, b) ((x>=a)&&(x<=b))

TouchScreen ts = TouchScreen(XP, YP, XM, YM, 600);

#define TFT_CS 10
#define TFT_DC 9
Adafruit_ILI9341 tft = Adafruit_ILI9341(TFT_CS, TFT_DC);

DS3231 rtc(SDA, SCL);

String dailyStuff[7] = {"\n agenda\n coat\n umbrella",
                       "\n wallet\n documents",
                       "\n gym bag\n dog treats",
                       "\n files\n pen drive",
                       "\n cell phone\n passport", 
                       "\n car keys\n bags",
                       "\n art supplies\n laptop charger"};

// define variables for if he has everything and the new and old strings
bool hasEverything = false;
String old = "";
String updated = "";
String printed = updated;
String add = "";
String tomorrowString = "";
String tb = "";
int day;

//define LED and motion sensor pins
const int MOTION_SENSOR = 3;
const int LED = 4;
// set boolean for startup
bool remind = false;

int index = 0;

int bx;
int by;

bool scrollUp;
bool scrollDown;
bool adding;

const char textLimit = 25;
char MyBuffer[textLimit];

#define FRAME_X 210
#define FRAME_Y 180
#define FRAME_W 50
#define FRAME_H 40

#define GREENBUTTON_X (FRAME_X + FRAME_W)
#define GREENBUTTON_Y FRAME_Y
#define GREENBUTTON_W FRAME_W
#define GREENBUTTON_H FRAME_H

void drawFrame() {
  tft.drawRect(GREENBUTTON_X, GREENBUTTON_Y + 7, FRAME_W, FRAME_H, ILI9341_BLACK);
}

void addMode() {
  keyboard();
  while (adding) {
    GetKeyPress(MyBuffer);
  }
  yesBtn();
}

// updates the string needed for the day based on the time
void dayUpdate() {
  if (rtc.getTime().hour == 23 && rtc.getTime().min == 58) {
    tb = tomorrowString;
    day = rtc.getTime().dow;
    updated = dailyStuff[day] + tb;
    tomorrowString = "";
  }
}

void checkScroll() {
  TSPoint p = ts.getPoint();
  bx = map(p.y, TS_MINX, TS_MAXX, 0, tft.width());
  by = map(p.x, TS_MINY, TS_MAXY, tft.height(),0);
  if ((260 < bx  && bx < 300) && (65 < by && by < 100)) {
    scrollUp = true;
  }
  else if ((260 < bx && bx < 300) && (100 < by && by < 135)) {
    scrollDown = true;
  }
  if (scrollDown) {
    index = updated.indexOf("\n ");
    if (index == -1) {
      printed = updated.substring(index + 2);
      tft.println(printed);
    }
    scrollDown = false;
  }

  if (scrollUp) {
    index = updated.indexOf(printed);
    if (index != 0) {
      printed = updated.lastIndexOf("\n ", index);
      tft.println(printed);
    }
    scrollUp = false;
  }
}

//arrays for keyboard
const char Mobile_KB[3][13] PROGMEM = {
  {0, 13, 10, 'Q', 'W', 'E', 'R', 'T', 'Y', 'U', 'I', 'O', 'P'},
  {1, 12, 9, 'A', 'S', 'D', 'F', 'G', 'H', 'J', 'K', 'L'},
  {3, 10, 7, 'Z', 'X', 'C', 'V', 'B', 'N', 'M'},
};

const char Mobile_NumKeys[3][13] PROGMEM = {
  {0, 13, 10, '1', '2', '3', '4', '5', '6', '7', '8', '9', '0'},
  {0, 13, 10, '-', '/', ':', ';', '(', ')', '$', '&', '@', '"'},
  {5, 8, 5, '.', '\,', '?', '!', '\''}
};

const char Mobile_SymKeys[3][13] PROGMEM = {
  {0, 13, 10, '[', ']', '{', '}', '#', '%', '^', '*', '+', '='},
  {4, 9, 6, '_', '\\', '|', '~', '<', '>'}, //4
  {5, 8, 5, '.', '\,', '?', '!', '\''}
};

//function that actually makes the keyboard look like a keyboard (has buttons and prints letters)
void MakeKB_Button(const char type[][13])
{
  tft.setTextSize(2);
  tft.setTextColor(ILI9341_WHITE);
  for (int y = 0; y < 3; y++)
  {
    int ShiftRight = 15 * pgm_read_byte(&(type[y][0]));
    for (int x = 3; x < 13; x++)
    {
      if (x >= pgm_read_byte(&(type[y][1]))) break;

      drawButton(15 + (30 * (x - 3)) + ShiftRight, 100 + (30 * y), 20, 25); // this will draw the button on the screen by so many pixels
      tft.setCursor(20 + (30 * (x - 3)) + ShiftRight, 105 + (30 * y));
      tft.print(char(pgm_read_byte(&(type[y][x]))));
    }
  }
  //ShiftKey
  drawButton(15, 160, 35, 25);
  tft.setCursor(27, 168);
  tft.print('^');

  //Special Characters
  drawButton(15, 190, 35, 25);
  tft.setCursor(21, 195);
  tft.print(F("SP"));

  //BackSpace
  drawButton(270, 160, 35, 25);
  tft.setCursor(276, 165);
  tft.print(F("BS"));

  //Return
  drawButton(270, 190, 35, 25);
  tft.setCursor(276, 195);
  tft.print(F("RT"));

  //Spacebar
  drawButton(60, 190, 200, 25);
  tft.setCursor(105, 195);
  tft.print(F("SPACE BAR"));
}

//function that draws buttons
void drawButton(int x, int y, int w, int h)
{
  // grey
  tft.fillRoundRect(x - 3, y + 3, w, h, 3, ILI9341_LIGHTGREY); //Button Shading

  // white
  tft.fillRoundRect(x, y, w, h, 3, ILI9341_WHITE);// outter button color

  //red
  tft.fillRoundRect(x + 1, y + 1, w - 1 * 2, h - 1 * 2, 3, ILI9341_RED); //inner button color
}

void GetKeyPress(char * textBuffer)
{
  static bool shift = false, special = false, back = false, lastSp = false, lastSh = false;
  static int bufIndex = 0;
  TSPoint p = ts.getPoint();
  Serial.print("bx = "); Serial.print(bx); Serial.print(" by = "); Serial.println(by);
  Serial.println(p.z);
  Serial.println("we're here");
  if (p.z >= 0)
  {
    //ShiftKey
    if (TouchButton(-10, 160, 30, 25))
    {
      shift = !shift;
      delay(200);
    }

    //Special Characters
    if (TouchButton(-10, 190, 30, 25))
    {
      special = !special;
      delay(200);
    }

    if (special != lastSp || shift != lastSh) //Start of changing the keys and make it look different
    {
      if (special)
      {
        if (shift)
        {
          tft.fillScreen(ILI9341_BLUE);
          MakeKB_Button(Mobile_SymKeys);
        }
        else
        {
          tft.fillScreen(ILI9341_BLUE);
          MakeKB_Button(Mobile_NumKeys);
        }
      }
      else
      {
        tft.fillScreen(ILI9341_BLUE);
        MakeKB_Button(Mobile_KB);
        tft.setTextColor(0xffff, 0xf800);
      }

      if (special)
        tft.setTextColor(0x0FF0, 0xf800);
      else
        tft.setTextColor(0xFFFF, 0xf800);

      tft.setCursor(21, 195);
      tft.print(F("SP"));

      if (shift)
        tft.setTextColor(0x0FF0, 0xf800);
      else
        tft.setTextColor(0xffff, 0xf800);

      tft.setCursor(27, 168);
      tft.print('^');

      lastSh = shift;
      lastSp = special;
      lastSh = shift;
    }//End of changing the keys and make it look different

    for (int y = 0; y < 3; y++)
    {
      int ShiftRight;
      if (special)
      {
        if (shift) //special+shift = symbol
          ShiftRight = 15 * pgm_read_byte(&(Mobile_SymKeys[y][0]));
        else //special only = number
          ShiftRight = 15 * pgm_read_byte(&(Mobile_NumKeys[y][0]));
      }
      else //alphabet
        ShiftRight = 15 * pgm_read_byte(&(Mobile_KB[y][0]));

      for (int x = 3; x < 13; x++)
      {
        if (x >=  (special ? (shift ? pgm_read_byte(&(Mobile_SymKeys[y][1])) : pgm_read_byte(&(Mobile_NumKeys[y][1]))) : pgm_read_byte(&(Mobile_KB[y][1])) )) break;

        if (TouchButton((33 * (x - 3)) + ShiftRight - 14, 100 + (30 * y), 20, 25)) // this will draw the button on the screen by so many pixels
        {
          if (bufIndex < (textLimit - 1))
          {
            delay(200);

            if (special)
            {
              if (shift)
                textBuffer[bufIndex] = pgm_read_byte(&(Mobile_SymKeys[y][x]));
              else
                textBuffer[bufIndex] = pgm_read_byte(&(Mobile_NumKeys[y][x]));
            }
            else
              textBuffer[bufIndex] = (pgm_read_byte(&(Mobile_KB[y][x])) + (shift ? 0 : ('a' - 'A')));

            bufIndex++;
          }
          break;
        }
      }
    }

    //Spacebar
    if (TouchButton(35, 190, 200, 25))
    {
      textBuffer[bufIndex++] = ' ';
      delay(200);
    }

    //BackSpace
    if (TouchButton(268, 160, 30, 25))
    {
      if ((bufIndex) > 0)
        bufIndex--;
      textBuffer[bufIndex] = 0;
      tft.setTextColor(0, ILI9341_BLUE);
      tft.setCursor(15, 80);
      tft.print(F("                          "));
      delay(200);
    }

    //Return
    if (TouchButton(268, 190, 30, 25))
    {
      tomorrowString = tomorrowString + textBuffer + "\n ";
      Serial.println(textBuffer);
      Serial.println("we got here too");
      adding = false;
      while (bufIndex > 0)
      {
        bufIndex--;
        textBuffer[bufIndex] = 0;
      }

      tft.setTextColor(0, ILI9341_BLUE);
      tft.setCursor(15, 80);
      tft.print(F("                         "));
      Serial.println(tomorrowString);
      yesBtn();
    }
  }
  tft.setTextColor(0xffff, 0xf800);
  tft.setCursor(15, 80);
  tft.print(textBuffer);
}

//Makes you know if the point that you touched is in the range computed by x,y,w,h
byte TouchButton(int x, int y, int w, int h)
{
  int X, Y;
  // Retrieve a point
  TSPoint p = ts.getPoint();
  Y = map(p.x, TS_MINY, TS_MAXY, tft.height(), 0);
  X = map(p.y, TS_MINX, TS_MAXX, 0, tft.width());

  return (IsWithin(X, x, x + w) & IsWithin(Y, y, y + h));
}

void keyboard() {
  tft.fillScreen(ILI9341_BLUE);
  MakeKB_Button(Mobile_KB);
  GetKeyPress(MyBuffer);
}

void yesBtn()
{
  tft.fillScreen(ILI9341_BLUE);
  //Asking Jim text
  tft.setCursor(10, 10);
  tft.setTextColor(ILI9341_WHITE);
  tft.setTextSize(2);
  tft.println("Hey Jim! Do you have      what you need for today?");
  //Printing item list
  tft.setTextSize(2);
  tft.println(updated);
  //Scroll buttons
  tft.fillRect(GREENBUTTON_X + 6, 60, GREENBUTTON_W - 12, 70, ILI9341_LIGHTGREY);
  tft.drawRect(GREENBUTTON_X + 6, 60, GREENBUTTON_W - 12, 35, ILI9341_BLACK);
  tft.drawRect(GREENBUTTON_X + 6, 95, GREENBUTTON_W - 12, 35, ILI9341_BLACK);
  //Scroll up arrow
  tft.setCursor(GREENBUTTON_X + 17, 65);
  tft.setTextColor(ILI9341_BLACK);
  tft.setTextSize(3);
  tft.println((char)24);
  //Scroll down arrow
  tft.setCursor(GREENBUTTON_X + 17, 100);
  tft.setTextColor(ILI9341_BLACK);
  tft.setTextSize(3);
  tft.println((char)25);
  //Add item button
  tft.fillRect(GREENBUTTON_X, 138, GREENBUTTON_W, GREENBUTTON_H, ILI9341_DARKGREEN);
  tft.drawRect(GREENBUTTON_X, 138, GREENBUTTON_W, GREENBUTTON_H, ILI9341_BLACK);
  tft.setCursor(GREENBUTTON_X + 7, 150);
  tft.setTextColor(ILI9341_WHITE);
  tft.setTextSize(2);
  tft.println("ADD");
  //YES button
  tft.fillRect(GREENBUTTON_X, GREENBUTTON_Y + 7, GREENBUTTON_W, GREENBUTTON_H, ILI9341_RED);
  drawFrame();
  tft.setCursor(GREENBUTTON_X + 7 , GREENBUTTON_Y + (GREENBUTTON_H / 2));
  tft.setTextColor(ILI9341_WHITE);
  tft.setTextSize(2);
  tft.println("YES");
}

void byeBtn()
{
  //Have a good day text
  tft.fillScreen(ILI9341_BLUE);
  tft.setCursor(10, 10);
  tft.setTextColor(ILI9341_WHITE);
  tft.setTextSize(4);
  tft.println("Great! Have  a good day");
  //bye button
  tft.fillRect(GREENBUTTON_X, GREENBUTTON_Y + 7, GREENBUTTON_W, GREENBUTTON_H, ILI9341_GREEN);
  drawFrame();
  tft.setCursor(GREENBUTTON_X + 7 , GREENBUTTON_Y + (GREENBUTTON_H / 2));
  tft.setTextColor(ILI9341_WHITE);
  tft.setTextSize(2);
  tft.println("BYE");
}

void setup() {
  // Set up serial monitor
  Serial.begin(9600);
  pinMode(MOTION_SENSOR, INPUT);
  pinMode(LED, OUTPUT);
  tft.begin();
  rtc.begin();
  // Set up the screen at first
  tft.setRotation(1);
  yesBtn();
}

void loop() {

  dayUpdate();
  
  if (digitalRead(MOTION_SENSOR) == HIGH) {
    digitalWrite(LED, HIGH);
    //Serial.println("motion sensor triggered");
    remind = true;
  }
  else if (digitalRead(MOTION_SENSOR) == LOW) {
    digitalWrite(LED, LOW);
    remind = false;
  }

  if (remind == false) {
    // fills the screen with black to save power 
    // and also not draw attention when not needed
    tft.fillScreen(ILI9341_BLACK);
    while (remind == false) {
      if (digitalRead(MOTION_SENSOR) == HIGH) {
        digitalWrite(LED, HIGH);
        //Serial.println("motion sensor triggered");
        remind = true;
      }
      else if (digitalRead(MOTION_SENSOR) == LOW) {
        digitalWrite(LED, LOW);
        remind = false;
      } 
    }
  }

  //only do things if the motion sensor gets tripped
  if (remind == true) {
    // first read the screen to see if he did anything
  // Retrieve a point
  TSPoint p = ts.getPoint();

  //See if there's any touch data
  if (p.z > MINPRESSURE && p.z < MAXPRESSURE) {
    //Scale using the calibration #'s and rotate coordinate system
    bx = map(p.y, TS_MINX, TS_MAXX, 0, tft.width());
    by = map(p.x, TS_MINY, TS_MAXY, tft.height(),0);
    if (hasEverything == false) {
      // if he presses the button
      if ((255 < bx && bx < 310) && (192 < by && by < 230)) {
        // set hasEverything to true and execute that part of the code
        hasEverything = true;
      }
      if (hasEverything == true) {
      byeBtn();
      delay(5000);
      hasEverything = false;
      yesBtn();
  }
    }
  }

   p = ts.getPoint();

  //See if there's any touch data
  if (p.z > MINPRESSURE && p.z < MAXPRESSURE) {
    //Scale using the calibration #'s and rotate coordinate system
    bx = map(p.y, TS_MINX, TS_MAXX, 0, tft.width());
    by = map(p.x, TS_MINY, TS_MAXY, tft.height(),0);
    Serial.print("bx = "); Serial.print(bx); Serial.print(" by = "); Serial.println(by);

    // pressed add button
    if (255 < bx && bx < 310 && 143 < by && by < 181) {
      bx = 10;
      by = 10;
      adding = true;
      addMode();
    }
    checkScroll();
  }

  // if he adds stuff from serial monitor
  if (Serial.available() > 0) {
    //// upload it onto the main display
    add = Serial.readStringUntil('\n');
    updated = updated + '\n' + ' ' + add;
    Serial.println(updated);
    yesBtn();
  }

  }

  
}

NOTE: After it is tested and confirmed to work with the Arduino, we will be adding in the IoT version of this code, which connects with Google API to return events without the need for input.

Schematic and Design Files

Schematic of the box at final demo