Vibrotactile Sensations: Exploring their impact on human body

Problem: Many people today experience an enormous amount of stress due to the fast-paced and competitive rhythms of a society that tries to survive under the uncertainty that Covid-19 brought. Stress constitutes a psycho-emotional condition that most of the times breaks out through the human body using various forms: headaches, stomach pains, fast pulses and breathing, sometimes even inertia. Many people turn to medicine drugs in an effort to find a way out, neglecting the therapeutic effects that a simple touch may have on their body. Alternative ways of medicine and well being such as pain and stress and management through meditation, therapeutic massage and reflexology are being neglected by the variety of people, especially in West societies.  Sometimes, a simple touch and push between your eyes, may prove to be enough to stop a strong headache or force yourself to release significant stress and feel asleep.

Solution: For my final project, I propose the design and computation of an e-bandage that can be wrapped around different areas of the human body (head, stomach, arm) and offer different vibrotactile sensations, that can either calm stress, treat pain or even induce an alert. In this project, I am about to explore different effects of vibrations and their impact on the human body, by playing with factors such as the spatial configuration and distance among the vibrators, their intensity, frequency and location upon the body. I intent to use two arrays of LRA vibration motors, where different combinations of motors are going to be activated for different haptic patterns. A touchpad matrix is going to be used for choosing which effect should be executed.

Components:

  • 8 x LRA vibrating motors
  • 8 x Transistors to increase power output of the motor
  • 8 x Adafruit haptic motor drivers or 8 x SparkFun Haptic Motor Driver (?)
  • conductive thread
  • optional: Flora microcontroller or LilyPad arduino

Fight Alzheimer’s Disease with Karaoke

Problem

Alzheimer’s Disease is an irreversible, progressive brain disorder that slowly destroys memory and thinking skills, and, eventually, the ability to carry out the simplest tasks. Experts suggest that more than 5.5 million Americans, most of them age 65 or older, may have dementia caused by Alzheimer’s. Recent estimates indicate that the disorder is ranked third in the leading cause of death in U.S. With longer life expectancy and larger population, prevention of Alzheimer’s Disease is no doubt challenging but also necessary.

So far little is known about the cause of Alzheimer’s Disease, and no evidence has proven any treatment that works for sure. However, the National Academies of Sciences, Engineering, and Medicine (NASEM),  found “encouraging but inconclusive” evidence for three types of interventions:

  •  Increased physical activity
  • Blood Pressure Control
  • Cognitive training.

Further study shows that informal cognitively stimulating activities, such as reading or playing games, may lower risk of Alzheimer’s-related cognitive impairment and dementia. In addition, social activity and physical activities also have positive impact for prevention.

Solution

I was trying to come up with a device that can encourage the person to engage in social activity, cognitively stimulating activity, and physical activity. Then I thought about the Pokemon Go which you get rewarded by going to different places; I also thought about video games where items can be shared across different devices/accounts when they are connected, which encourages players to play together.

The device I made is basically a karaoke device that can display lyrics to the screen. An optional microphone is connected with the device through Bluetooth.

The layout of the device
The optional Microphone gives the vibe of Karaoke.

Learning the song and memorizing lyrics is also an informal cognitively stimulating activity. When the reader has learnt the song and does not need the lyrics anymore, the lyrics can be switched off by the top button.

However, there is no song to play initially. The user can select a music genre and time period once per day to obtain a random song that is in the selected category; to earn more songs, the user has to do some physical exercise such as walking and running for a period . The physical activity is detected by an accelerometer placed in the users’ shoes.

To encourage social activity, songs can be “shared” between two users. For example, if I find out that my friend has a song that I would like to have on my device, he can share the song with me. However, it is limited to one shared song per day. There is also a PK feature that allows the user to challenge each other; the two users will sing the same song, and whoever look at the lyrics for the least amount of time wins. The purpose of these mechanism is to make users meet more with their families and friends.

Proof of Logic

Layout

The layout of the components of the prototype

For this prototype, two Arduinos are used; The one on the left in the photo above is the prototype of the device, and the one on the right is a “dummy device” that only sends information through the transceiver. The “dummy device” is needed to show the interaction between two device; it is not made into another full prototype because of the lack of materials.

The Arduino of the full prototype connects to a accelerometer, a potentiometer, a mini-mp3 player, 4 buttons, a LCD screen, and a transceiver. The accelerometer detects motion for physical exercise. The potentiometer is for volume control. The mini-mp3 player works with the transducer speaker to play song. The buttons are for the user interface. The LCD screen is to show the user interface and display lyrics when songs are playing. The Transceiver allows communication between the prototype and the “dummy” Arduino.

The “dummy” Arduino is connected to a button and a transceiver; pressing the button triggers the Arduino to send information through the transceiver. The “dummy” Arduino can be powered by an external  of a 9V battery. However, it is plugged into PC during testing for convenience of uploading codes.

Interaction

Interface

The user interface of the prototype includes a LCD screen and four buttons. The first button from the top represents rolling the rotary encoder up, and the second button represents rolling the rotary encoder down. The rotary encoder was labeled “rotator” in the sketch of the layout of the device. The third button is the “Confirm” button of the rotary encoder, and the fourth button is the “Back/Lyrics” button.

In the following video, the functions of the four buttons are shown by going through the menu. In menu, there are 3 modules to choose: “Unlocked Songs”, “Be Connected”, and “Jackpot”.  The first and second button switches the current option.  The “Confirm” button select the selects option. And the “Back” button goes back to the previous stage. Notice that when option “Unlocked Songs” was selected in the video, it said empty. It was because there was no unlocked songs yet.

Jackpot

The next video shows the option “Jackpot”. The user has to select the desired genre and times of the song, then a random song from the selected category will be unlocked. For demonstration purpose, there are only 3 genres (Movie soundtrack, Pop, Jazz) and 2 period (50s, 2010s). Again, the user can only unlock one song per day through jackpot.

Physical Exercise

If the user wants to obtain more songs in a day, he/she can do so by doing physical exercise. An accelerometer can be placed in the user’s shoe or on user’s wrist as a fit-band; the physical activity will be detected by the accelerometer. I shook the accelerometer in the video to stimulate the physical activity. After exercising, the user obtains a chance to unlock a song again.

Playing Unlocked Song

With songs unlocked from the “Jackpot”, the user can play the unlocked songs and sing along them. The potentiometer represents the volume control of the device. When the song is being played, the “Confirm” button pauses the song and the “Back/ Lyrics” button toggles the lyrics. The lyrics are coded into the prototype because there are better applications to do that.  In addition, the potentiometer used in the demonstration does not always have a stable contact with the breadboard, which overwhelms the Arduino with data and makes the buttons less responsive.

Receive Song

If the user stays in the module of “Be Connected” and switches to “Share Song” as the current option (but not selecting it), the device can receive a new song from a different device. However, each device can only receive a song from another device once per day. After a receipt, no new song will be added anymore.

Share Song

In the “Be Connected” module, if the user selects the option of “Share Song” as shown in the video, the user can then choose a n unlocked song to share to a friend. There is no limits on how many songs a device can share per day.

Compete

In the module of “Be Connected”, the user can select an unlocked song to challenge his/her friend. When the song is being played, the device keeps track of how long the lyrics is been looked at. When the song finished, the devices will compare the time of showing the lyrics and decide a winner. In the video, the dummy Arduino sends its time of displaying the lyrics after the button was pressed.

Final Notes

  • When I started to work on the transceiver, I found out that I did not have enough female-to-male jumper wires. I bought some immediately but they only came the night before the crit, which makes it more stressful than it should be. Next time when I am preparing the components I have to not only think about major devices but all the materials including wires.
  • There are definitely more rooms for improvement. First of all, the device is designed as a device that downloads music from the internet instead of storing music in a micro-SD card. Some wifi or bluetooth parts should be able to connect the device to the internet. Secondly, the code can be futher optimized. It is cumbersome and inefficient at the point. Thirdly, more games such as guessing the name of the song can be added in addition to the competition of sing without looking at the lyrics. Fourthly, a vibration motor can be added to the device to notify the user when the opponent started to look at lyrics in competition. In addition, the interface can be better with lively sound effects added. For example, before a new song is unlocked, the device can play a little bit of drum rolling. Finally, the device can only became a real karaoke device when a microphone is incorporated.

Schematics

Schematics for the full prototype
Schematics for the dummy Arduino

Code

Code for the full prototype:

//Library
#include "Arduino.h"
#include "SoftwareSerial.h"
#include "DFRobotDFPlayerMini.h"
#include <Wire.h>
#include <LiquidCrystal_I2C.h>
#include <SPI.h>
#include <nRF24L01.h>
#include <RF24.h>

//Set up of hardwares
//Mp3 player
SoftwareSerial mySoftwareSerial(6, 7); // RX, TX
DFRobotDFPlayerMini myDFPlayer;
void printDetail(uint8_t type, int value);
//LCD
LiquidCrystal_I2C screen(0x27, 16, 2);
//Transceiver
RF24 radio(8, 9); // CE, CSN



//Global Variables
//Constant variables
const int ROTARY_UP = 5;
const int ROTARY_DOWN = 4;
const int ROTARY_CONFRIM = 3;
const int BACK_LYRICS = 2;
const int VOLUME_CONTROL = A0;
const int XPIN = A1;
const int YPIN = A2;
const int ZPIN = A3;

//Chaning Variables
//Button
int ButtonState_rotaryUp;
int ButtonState_rotaryDown;
int ButtonState_Confirm;
int ButtonState_Back_Lyrics;

int lastButtonState_rotaryUp = HIGH;
int lastButtonState_rotaryDown = HIGH;
int lastButtonState_Confirm = HIGH;
int lastButtonState_Back_Lyrics = HIGH;

//Mp3
int lastVolume = 15;

//Debounce Varibale
unsigned long lastDebounceTime = 0;  // the last time the output pin was toggled
unsigned long debounceDelay = 50;    // the debounce time; increase if the output flickers

//The variables related to display the lyrics
bool lyrics = false;
//This is for counting the time when lyrics are displayed throughout the song.
unsigned long lyrics_timer_Start;
unsigned long lyrics_time;
unsigned long lyrics_time_oponent;
bool playing = false;

//Accelerometer
bool timer = false;
unsigned long accelerometer_timer_Start;
unsigned long accelerometer_time;
//Exercise for 10 seconds to get a chance for jackpot
const unsigned long accelerometer_time_jackpot = 10000;

//The variables used in building the interface
String interFace = "Menu";
String interFace_Secondary;
int interFace_Option = 1;

String selected_Genre;
String selected_Period;

int Jackpot_Chance = 1;
unsigned long Jackpot_Clock;
const int OneDay = 60000;         //For demo purpose 1_Day is 1 minute.

unsigned long Receive_Clock;
bool song_received = false;

//Can't read the number of file yet in the global scope so make the array as big as the capacity of the SD card. Or assume the information is already given when the songs are downloaded from internet.
//The sizes of the array are set for the demo purpose
//Arrary that includes all the unlocked songs
int unlocked_Song[5];
int input_indice_unlocked;
//Array taht includes the unlocked 2010s movie soundtrack
int Unlocked_2010s_Movie[1];
int input_indice_movie;
int Unlocked_2010s_Pop[1];
int input_indice_pop;
int Unlocked_50s_Jazz[3];
int input_indice_jazz;

//Transceiver's addresses
//One for writing and one for reading
const byte addresses[][6] = {"00001", "00002"};


//Function
//Functions related to the MP3 player
void read_information() {
  Serial.println(myDFPlayer.readState()); //read mp3 state
  Serial.println(myDFPlayer.readVolume()); //read current volume
  Serial.println(myDFPlayer.readEQ()); //read EQ setting
  Serial.println(myDFPlayer.readFileCounts()); //read all file counts in SD card
  Serial.println(myDFPlayer.readCurrentFileNumber()); //read current play file number
  //Serial.println(myDFPlayer.readFileCountsInFolder(3)); //read fill counts in folder SD:/03
}

//Function related to the transveriver
void share_song(int shared_song) {
  radio.stopListening();
  radio.write(&shared_song, sizeof(shared_song));
}

//Function that receives song from  "Be connected"
void receive_song() {
  radio.startListening();
  if (radio.available() && song_received == false ) {
    int shared_song;
    radio.read(&shared_song, sizeof(shared_song));
    //Write the number of the number into the array for unlocked_songs
    unlocked_Song[input_indice_unlocked] = shared_song;
    input_indice_unlocked += 1;
    song_received = true;
    screen.clear();
    screen.setCursor(4, 0);
    screen.print("New Song");
    screen.setCursor(5, 1);
    screen.print("Added!");
    delay(3000);
    screen.clear();
    screen.setCursor(6, 0);
    screen.print("Menu");
    screen.setCursor(1, 1);
    screen.print("Unlocked Songs");
    interFace = "Menu";
    interFace_Option = 1;
    radio.stopListening();
  }
}



//Function related to the accelerometer
void read_accelerometer() {
  if (timer == false) {
    if (analogRead(ZPIN) > 321) {
      timer = true;
      accelerometer_timer_Start = millis();
    }
  }
  else {
    if (analogRead(ZPIN) <= 321 && analogRead(ZPIN) >= 317) {
      accelerometer_time += millis() - accelerometer_timer_Start;
      timer = false;
    }
  }
}

//Functions related to the interface of the device
//Function that checks whether there is an zero in the array, which means there is still song for that category to unlock. (supplemental to function jackpot)
bool CheckforZero(String Genre, String Period) {
  if (Genre == "Movie" && Period == "2010s") {
    for (int i = 0; i < (sizeof(Unlocked_2010s_Movie) / sizeof(Unlocked_2010s_Movie[0])); i++) {
      if (Unlocked_2010s_Movie[i] == 0) {
        return true;
      }
    }

  }


  else if (Genre == "Pop" && Period == "2010s") {
    for (int i = 0; i < (sizeof(Unlocked_2010s_Pop) / sizeof(Unlocked_2010s_Pop[0])); i++) {
      if (Unlocked_2010s_Pop[i] == 0) {
        return true;
      }
    }
  }

  else if (Genre == "Jazz" && Period == "50s") {
    for (int i = 0; i < (sizeof(Unlocked_50s_Jazz) / sizeof(Unlocked_50s_Jazz[0])); i++) {
      if (Unlocked_50s_Jazz[i] == 0) {
        return true;
      }
    }
  }

  return false;
  //return false when there is no more 0 in the array


}


//Function that check whether a number is already in an array. (supplemental to function jackpot)
bool checkNumberinList(int numb, String Genre, String Period) {
  if (Genre == "Movie" && Period == "2010s") {
    //Serial.println((sizeof(Unlocked_2010s_Movie) / sizeof(Unlocked_2010s_Movie[0])));
    //Serial.println(Unlocked_2010s_Movie[0]);
    for (int i = 0; i < (sizeof(Unlocked_2010s_Movie) / sizeof(Unlocked_2010s_Movie[0])); i++) {
      if (Unlocked_2010s_Movie[i] == numb) {
        //Serial.println("Here");
        return true;
      }
    }
  }

  else if (Genre == "Pop" && Period == "2010s") {
    for (int i = 0; i < (sizeof(Unlocked_2010s_Pop) / sizeof(Unlocked_2010s_Pop[0])); i++) {
      if (numb == Unlocked_2010s_Pop[i]) {
        return true;
      }
    }
  }

  else if (Genre == "Jazz" && Period == "50s") {
    for (int i = 0; i < (sizeof(Unlocked_50s_Jazz) / sizeof(Unlocked_50s_Jazz[0])); i++) {
      if (numb == Unlocked_50s_Jazz[i]) {
        return true;
      }
    }
  }
}



//Function that select a random song to unlock in Jackpot. (supplemental to function button pressed)
int jackpot(String Genre, String Period) {
  //Check if the selected category still has songs in the songbank to unlock
  if (CheckforZero(Genre, Period) != true) {
    //Print the information on the lcd screen
    screen.clear();
    screen.setCursor(1, 0);
    screen.print(String("No more " + Genre));
    screen.setCursor(3, 1);
    screen.print(String("in " + Period));
    delay(5000);
    screen.setCursor(1, 0);
    screen.print("Select another");
    screen.setCursor(2, 1);
    screen.print("genre/period");
    delay(5000);
    screen.clear();

    return 0;
  }

  else {

    if (Genre == "Movie" && Period == "2010s") {
      //Number 1 to number 1 represents Movie Soundtracks in 2010s
      int randomPick = random(1, 2);

      while (checkNumberinList(randomPick, Genre, Period) == true) {
        randomPick = random(1, 2);
        //Serial.println(randomPick);
      }
      //Add the song into the array of unlocked song
      unlocked_Song[input_indice_unlocked] = randomPick;
      input_indice_unlocked += 1;

      //Add the song into the array of unlcoked movie soundtrack array
      Unlocked_2010s_Movie[input_indice_movie] = randomPick;
      input_indice_movie += 1;


      return randomPick;
    }


    else if (Genre == "Pop" && Period == "2010s") {
      //Number 1 to number 1 represents Movie Soundtracks in 2010s
      int randomPick = random(2, 3);
      while (checkNumberinList(randomPick, Genre, Period) == true) {
        randomPick = random(2, 3);
      }
      //Add the song into the array of unlocked song
      unlocked_Song[input_indice_unlocked] = randomPick;
      input_indice_unlocked += 1;

      //Add the song into the array of unlcoked movie soundtrack array
      Unlocked_2010s_Pop[input_indice_pop] = randomPick;
      input_indice_pop += 1;

      return randomPick;
    }

    else if (Genre == "Jazz" && Period == "50s") {
      //Number 1 to number 1 represents Movie Soundtracks in 2010s
      int randomPick = random(3, 6);
      while (checkNumberinList(randomPick, Genre, Period) == true) {
        randomPick = random(3, 6);
      }
      //Add the song into the array of unlocked song
      unlocked_Song[input_indice_unlocked] = randomPick;
      input_indice_unlocked += 1;

      //Add the song into the array of unlcoked movie soundtrack array
      Unlocked_50s_Jazz[input_indice_jazz] = randomPick;
      input_indice_jazz += 1;

      return randomPick;
    }
  }
}


//Function thats returns the name of the song given the interface_option. (supplemental to function button pressed)
String SongName(int song) {
  if (song == 1) {
    return "City of Star";
  }
  else if (song == 2) {
    return "Stay";
  }
  else if (song == 3) {
    return "Fly Me to the Moon";
  }
  else if (song == 4) {
    return "All of You";
  }
  else if (song == 5) {
    return "The Best is yet to Come";
  }
}

//Function that determines what happens when the rotary enocoder is rolled up or down. (supplemental to function button pressed)
void Rotary(int input, String pressed_button, int min, int max) {
  if (pressed_button == "ROTARY_UP") {
    if (input == 1) {
      interFace_Option = max;
    }

    else {
      interFace_Option = input - 1;
    }
  }

  else if (pressed_button == "ROTARY_DOWN") {
    if (input == max) {
      interFace_Option = 1;
    }

    else {
      interFace_Option = input + 1;
    }
  }
}


//Function that decide what to do when buttons are pressed. (supplemental to function Read Button)
void button_pressed(String pressed_button) {
  //When buttons are pressed at the menu
  if (interFace == "Menu") {
    Rotary(interFace_Option, pressed_button, 1, 3);

    if (pressed_button == "ROTARY_CONFIRM") {
      if (interFace_Option == 1) {
        interFace = "Unlocked Songs";
        interFace_Secondary = "Select_Song";
        interFace_Option = 1;

      }

      else if (interFace_Option == 2) {
        interFace = "Jackpot";
        //Check if the user has earned a chance for jackpot through exercising
        if (accelerometer_time >= accelerometer_time_jackpot) {
          Serial.println(accelerometer_time);
          Jackpot_Chance += 1;
          accelerometer_time = 0;
        }
        if (Jackpot_Chance == 0) {
          //No more chance to get a new song for the day through Jackpot
          interFace_Secondary = "NoMore";
        }
        else {
          interFace_Secondary = "Genre";
        }
        interFace_Option = 1;

      }

      else if (interFace_Option == 3) {
        interFace = "Be Connected";
        interFace_Option = 1;
        interFace_Secondary = "Receive";
      }
    }

    //Display the options in Menu to the LCD screen
    //Clear the screen first
    screen.clear();
    if (interFace == "Menu") {
      screen.setCursor(6, 0);
      screen.print("Menu");
      if (interFace_Option == 1) {
        screen.setCursor(1, 1);
        screen.print("Unlocked Songs");
      }
      else if (interFace_Option == 2) {
        screen.setCursor(4, 1);
        screen.print("Jackpot");
      }
      else if (interFace_Option == 3) {
        screen.setCursor(2, 1);
        screen.print("Be Connected");
      }
    }
    else if (interFace == "Unlocked Songs") {
      screen.setCursor(1, 0);
      screen.print("Unlocked Songs");
      if (unlocked_Song[interFace_Option - 1] == 0) {
        screen.setCursor(5, 1);
        screen.print("Empty");
        delay(3000);
        screen.clear();
        screen.setCursor(6, 0);
        screen.print("Menu");
        screen.setCursor(1, 1);
        screen.print("Unlocked Songs");
        interFace = "Menu";
        interFace_Option = 1;
      }

      else {
        int song = unlocked_Song[interFace_Option - 1];
        screen.setCursor((16 - SongName(song).length()) / 2, 1);
        screen.print(SongName(song));
      }
    }
    else if (interFace == "Jackpot") {
      if (interFace_Secondary == "NoMore") {
        screen.setCursor(2, 0);
        screen.print("Unavailable");
        screen.setCursor(3, 1);
        screen.print("Right now");
        delay(3000);
        screen.clear();
        screen.setCursor(1, 0);
        screen.print("Earn More Song");
        screen.setCursor(1, 1);
        screen.print("by exercising");
        delay(3000);
        screen.clear();
        screen.setCursor(2, 0);
        screen.print("and meeting");
        screen.setCursor(4, 1);
        screen.print("friends!");
        delay(3000);
        //Goes back the menu page
        screen.clear();
        interFace = "Menu";
        interFace_Option = 1;
        screen.setCursor(6, 0);
        screen.print("Menu");
        screen.setCursor(1, 1);
        screen.print("Unlocked Songs");

      }


      else  {
        screen.setCursor(5, 0);
        screen.print("Genre");
        screen.setCursor(5, 1);
        screen.print("Movie");
      }
    }

    else if (interFace == "Be Connected") {
      screen.setCursor(2, 0);
      screen.print("Be Connected");
      screen.setCursor(3, 1);
      screen.print("Share Song");
    }
  }

  //When buttons are pressed at Unlocked Songs
  else if (interFace == "Unlocked Songs") {
    if (interFace_Secondary == "Select_Song") {
      Rotary(interFace_Option, pressed_button, 1, input_indice_unlocked);


      if (pressed_button == "ROTARY_CONFIRM") {
        myDFPlayer.play(unlocked_Song[interFace_Option - 1]);

        interFace_Secondary = "Playing_song";
        playing = true;
        screen.clear();
      }

      else if (pressed_button == "BACK_LYRICS") {
        interFace = "Menu";
        interFace_Option = 1;
        screen.clear();
        screen.setCursor(6, 0);
        screen.print("Menu");
        screen.setCursor(1, 1);
        screen.print("Unlocked Songs");

      }
    }

    else if (interFace_Secondary == "Playing_song") {
      if (pressed_button == "ROTARY_CONFIRM") {
        playing = !playing;
        if (playing == false) {
          Serial.println("pause");
          myDFPlayer.pause();
        }
        else {
          myDFPlayer.start();
        }
      }

      else if (pressed_button == "BACK_LYRICS") {
        //Serial.println(myDFPlayer.readState());
        //This means the song is finished
        if (myDFPlayer.readState() == 512) {
          if (lyrics == true) {
            lyrics = false;
          }

          interFace = "Menu";
          interFace_Option = 1;
          screen.clear();
          interFace = "Menu";
          interFace_Option = 1;
          screen.setCursor(6, 0);
          screen.print("Menu");
          screen.setCursor(1, 1);
          screen.print("Unlocked Songs");
        }
        else {
          lyrics = !lyrics;
          //Serial.println(lyrics);
          if (lyrics == true) {

            screen.setCursor(5, 0);
            screen.print("Lyrics");
            screen.setCursor(5, 1);
            screen.print("Lyrics");
          }
          else {
            screen.clear();
            //Add the lyrics-displaying time to the sum

          }
        }
      }
    }

    //Display the options in Menu to the LCD screen
    //Clear the screen first

    if (interFace == "Unlocked Songs") {
      if (unlocked_Song[0] == 0) {
        screen.clear();
        screen.setCursor(1, 0);
        screen.print("Unlocked Songs");
        screen.setCursor(5, 1);
        screen.print("Empty");
      }

      else if (interFace_Secondary == "Select_Song") {
        int song = unlocked_Song[interFace_Option - 1];
        //Serial.println("Hello");
        screen.clear();
        screen.setCursor(1, 0);
        screen.print("Unlocked Songs");
        screen.setCursor((16 - SongName(song).length()) / 2, 1);
        screen.print(SongName(song));
      }
    }
    else if (interFace == "Menu") {
      screen.clear();
      screen.setCursor(6, 0);
      screen.print("Menu");
      if (interFace_Option == 1) {
        screen.setCursor(1, 1);
        screen.print("Unlocked Songs");
      }
    }
  }

  //When buttons are pressed at Jackpot
  else if (interFace == "Jackpot") {
    //Selecting the Genre of the music of jackpot
    if (interFace_Secondary == "Genre") {
      Rotary(interFace_Option, pressed_button, 1, 3);


      if (pressed_button == "ROTARY_CONFIRM") {
        if (interFace_Option == 1) {
          selected_Genre = "Movie";
        }

        else if (interFace_Option == 2) {
          selected_Genre = "Jazz";
        }

        else if (interFace_Option == 3) {
          selected_Genre = "Pop";
        }

        interFace_Secondary = "Period";
        interFace_Option = 1;
        //Serial.println(selected_Genre);

      }

      else if (pressed_button == "BACK_LYRICS") {
        interFace = "Menu";
        interFace_Option = 1;
      }

      //Display the options in Menu to the LCD screen
      //Clear the screen first
      screen.clear();
      if (interFace == "Menu") {
        screen.setCursor(6, 0);
        screen.print("Menu");
        screen.setCursor(1, 1);
        screen.print("Unlocked Songs");
      }
      else if (interFace_Secondary == "Period") {
        screen.setCursor(5, 0);
        screen.print("Period");
        screen.setCursor(5, 1);
        screen.print("2010s");
      }

      else {
        screen.setCursor(5, 0);
        screen.print("Genre");
        if (interFace_Option == 1) {
          screen.setCursor(5, 1);
          screen.print("Movie");
        }

        else if (interFace_Option == 2) {
          screen.setCursor(6, 1);
          screen.print("Jazz");
        }

        else if (interFace_Option == 3) {
          screen.setCursor(6, 1);
          screen.print("Pop");
        }
      }
    }

    //Selecting the Period
    else if (interFace_Secondary == "Period") {
      Rotary(interFace_Option, pressed_button, 1, 2);


      if (pressed_button == "ROTARY_CONFIRM") {
        if (interFace_Option == 1) {
          selected_Period = "2010s";
          //When there is no more songs in the selected category to unlock
        }
        else if (interFace_Option == 2) {
          selected_Period = "50s";
        }

        int song = jackpot(selected_Genre, selected_Period);
        if (song == 0) {
          //Go back to choose another genre
          interFace_Secondary = "Genre";
          interFace_Option = 1;
          screen.setCursor(5, 0);
          screen.print("Genre");
          screen.setCursor(5, 1);
          screen.print("Movie");
          return;
        }
        else {
          //Serial.println(jackpot(selected_Genre, selected_Period));
          Jackpot_Chance -= 1;
          screen.clear();
          screen.setCursor(4, 0);
          screen.print("Jackpot!");
          String Acquired_Song = SongName(song);
          screen.setCursor((16 - Acquired_Song.length()) / 2, 1);
          screen.print(Acquired_Song);
          delay(2000);
          //Returns to the Menu page
          interFace = "Menu";
          interFace_Option = 1;
          screen.clear();
          screen.setCursor(6, 0);
          screen.print("Menu");
          screen.setCursor(1, 1);
          screen.print("Unlocked Songs");
          return;
        }
      }

      else if (pressed_button == "BACK_LYRICS") {
        interFace_Secondary = "Genre";
        interFace_Option = 1;
      }

      //Display the options in Menu to the LCD screen
      //Clear the screen first
      screen.clear();
      if (interFace_Secondary == "Genre") {
        screen.setCursor(5, 0);
        screen.print("Genre");
        screen.setCursor(5, 1);
        screen.print("Movie");
      }
      else {
        screen.setCursor(5, 0);
        screen.print("Period");
        if (interFace_Option == 1) {
          screen.setCursor(5, 1);
          screen.print("2010s");
        }

        else if (interFace_Option == 2) {
          screen.setCursor(6, 1);
          screen.print("50s");
        }
      }
    }
  }

  //When the interface is Be Connected
  else if (interFace == "Be Connected") {
    if (interFace_Secondary == "Receive") {
      Rotary(interFace_Option, pressed_button, 1, 2);

      if (pressed_button == "ROTARY_CONFIRM") {
        if (interFace_Option == 1) {
          interFace_Secondary = "share_song";
          interFace_Option = 1;
        }

        else if (interFace_Option == 2) {
          interFace_Secondary = "compete";
          interFace_Option = 1;
        }
      }


      else if (pressed_button == "BACK_LYRICS") {
        interFace = "Menu";
        interFace_Option = 1;
      }

      //lcd display
      screen.clear();
      if (interFace == "Menu") {
        screen.clear();
        screen.setCursor(6, 0);
        screen.print("Menu");
        screen.setCursor(1, 1);
        screen.print("Unlocked Songs");
      }
      else if (interFace_Secondary == "Receive") {
        screen.setCursor(2, 0);
        screen.print("Be Connected");
        if (interFace_Option == 1) {
          screen.setCursor(3, 1);
          screen.print("Share Song");
        }
        else if (interFace_Option == 2) {
          screen.setCursor(4, 1);
          screen.print("Compete");
        }
      }

      else if (interFace_Secondary == "share_song") {
        screen.setCursor(3, 0);
        screen.print("Share Song");
        if (unlocked_Song[0] == 0) {
          screen.setCursor(5, 1);
          screen.print("Empty");
          delay(2000);
          interFace_Secondary = "Receive";
          interFace_Option = 1;
          screen.setCursor(2, 0);
          screen.print("Be Connected");
          screen.setCursor(3, 1);
          screen.print("Share Song");
        }

        else {
          int song = unlocked_Song[interFace_Option - 1];
          //Serial.println("Hello");
          screen.setCursor((16 - SongName(song).length()) / 2, 1);
          screen.print(SongName(song));
        }
      }

      else if (interFace_Secondary == "compete") {
        screen.setCursor(4, 0);
        screen.print("Compete");
        if (unlocked_Song[0] == 0) {
          screen.setCursor(5, 1);
          screen.print("Empty");
          delay(2000);
          interFace_Secondary = "Receive";
          interFace_Option = 1;
          screen.setCursor(2, 0);
          screen.print("Be Connected");
          screen.setCursor(3, 1);
          screen.print("Share Song");
        }

        else {
          int song = unlocked_Song[interFace_Option - 1];
          //Serial.println("Hello");
          screen.setCursor((16 - SongName(song).length()) / 2, 1);
          screen.print(SongName(song));
        }
      }







    }

    else if (interFace_Secondary == "Playing_song") {
      if (pressed_button == "ROTARY_CONFIRM") {
        playing = !playing;
        if (playing == false) {
          Serial.println("pause");
          myDFPlayer.pause();
        }
        else {
          myDFPlayer.start();
        }
      }

      else if (pressed_button == "BACK_LYRICS") {
        //Serial.println(myDFPlayer.readState());
        //This means the song is finished
        if (myDFPlayer.readState() == 512) {
          if (lyrics == true) {
            lyrics_time += millis() - lyrics_timer_Start;
            lyrics = false;
          }

          //Send and receive the lyrics_time, then determines who wins
          radio.stopListening();
          radio.write(&lyrics_time, sizeof(lyrics_time));
          radio.startListening();
          while (!radio.available()); //wait until the lyrics_time of the opponenet was received
          radio.read(&lyrics_time_oponent , sizeof(lyrics_time_oponent));

          if (lyrics_time <= lyrics_time_oponent) {
            screen.setCursor(4, 0);
            screen.print("You Win!");
            screen.setCursor(2, 1);
            screen.print("Great Work!");
          }
          else {
            screen.setCursor(4, 0);
            screen.print("You Lose");
          }
          delay(5000);
          lyrics_time = 0;
          lyrics_time_oponent = 0;

          //Goes back to the menu
          interFace = "Menu";
          interFace_Option = 1;
          screen.clear();
          interFace = "Menu";
          interFace_Option = 1;
          screen.setCursor(6, 0);
          screen.print("Menu");
          screen.setCursor(1, 1);
          screen.print("Unlocked Songs");
        }
        else {
          Serial.println("Lyrics");
          lyrics = !lyrics;
          //Serial.println(lyrics);
          if (lyrics == true) {
            lyrics_timer_Start = millis();
            screen.setCursor(5, 0);
            screen.print("Lyrics");
            screen.setCursor(5, 1);
            screen.print("Lyrics");
          }
          else {
            screen.clear();
            //Add the lyrics-displaying time to the sum
            lyrics_time += millis() - lyrics_timer_Start;
          }
        }
      }
    }
    //Instead of receiving information, either sharing song or competing
    else {
      Rotary(interFace_Option, pressed_button, 1, input_indice_unlocked);


      if (pressed_button == "ROTARY_CONFIRM") {
        //Sending the number of the song through transceiver
        if (interFace_Secondary == "share_song") {
          int song = unlocked_Song[interFace_Option - 1];
          share_song(song);
          //lcd displays the song shared confirmation
          //Serial.println("Here");
          screen.clear();
          screen.setCursor(6, 0);
          screen.print("Song");
          screen.setCursor(4, 1);
          screen.print("Shared!");
          delay(2000);
          interFace_Secondary = "Receive";
          interFace_Option = 1;
        }

        //For competing
        else if (interFace_Secondary == "compete") {
          int song = unlocked_Song[interFace_Option - 1];
          share_song(song);
          myDFPlayer.play(unlocked_Song[interFace_Option - 1]);

          interFace_Secondary = "Playing_song";
          playing = true;

          screen.clear();
        }
      }

      else if (pressed_button == "BACK_LYRICS") {
        interFace_Secondary = "Receive";
        interFace_Option = 1;
      }

      //lcd display
      screen.clear();
      if (interFace_Secondary == "Receive") {
        screen.setCursor(2, 0);
        screen.print("Be Connected");
        screen.setCursor(3, 1);
        screen.print("Share Song");
      }

      else if (interFace_Secondary != "Playing_song") {
        if (interFace_Secondary == "share_song") {
          screen.setCursor(3, 0);
          screen.print("Share Song");
        }
        else if (interFace_Secondary == "compete") {
          screen.setCursor(4, 0);
          screen.print("Compete");
        }
        int song = unlocked_Song[interFace_Option - 1];
        screen.setCursor((16 - SongName(song).length()) / 2, 1);
        screen.print(SongName(song));
      }
    }
  }
}


//Function that read the pressing of the buttons
void read_buttons() {
  //Read Potentiometer and adjust the volume
  int potVal = analogRead(VOLUME_CONTROL);
  int currentVolume = map(potVal, 0, 1023, 0, 30);

  if (currentVolume != lastVolume) {
    //Serial.println(currentVolume);
    myDFPlayer.volume(currentVolume);
    lastVolume = currentVolume;
  }

  //Reading the buttons (with debounce)
  int reading_Up = digitalRead(ROTARY_UP);
  int reading_Down = digitalRead(ROTARY_DOWN);
  int reading_Confirm = digitalRead(ROTARY_CONFRIM);
  int reading_back_Lyrics = digitalRead(BACK_LYRICS);

  if (reading_Up != lastButtonState_rotaryUp || reading_Down != lastButtonState_rotaryDown || reading_Confirm != lastButtonState_Confirm || reading_back_Lyrics != lastButtonState_Back_Lyrics) {
    // reset the debouncing timer
    lastDebounceTime = millis();
  }

  if ((millis() - lastDebounceTime) > debounceDelay) {
    // whatever the reading is at, it's been there for longer than the debounce
    // delay, so take it as the actual current state:

    //Rotary up button
    // if the button state has changed:
    if (reading_Up != ButtonState_rotaryUp) {
      ButtonState_rotaryUp = reading_Up;

      if (ButtonState_rotaryUp == LOW) {
        //Serial.println(interFace_Option);
        Serial.println("Rotary Up");
        button_pressed("ROTARY_UP");
        Serial.println(interFace_Option);
      }
    }

    //Rotary Down Button
    if (reading_Down != ButtonState_rotaryDown) {
      ButtonState_rotaryDown = reading_Down;

      if (ButtonState_rotaryDown == LOW) {
        Serial.println("Rotary Down");
        button_pressed("ROTARY_DOWN");
        Serial.println(interFace_Option);
      }
    }

    //Rotary Confirm Button
    if (reading_Confirm != ButtonState_Confirm) {
      ButtonState_Confirm = reading_Confirm;

      if (ButtonState_Confirm == LOW) {
        Serial.println("Confirm");
        button_pressed("ROTARY_CONFIRM");
        Serial.println(interFace);
      }
    }

    //Back or show lyric Button
    if (reading_back_Lyrics != ButtonState_Back_Lyrics) {
      ButtonState_Back_Lyrics = reading_back_Lyrics;

      if (ButtonState_Back_Lyrics == LOW) {
        Serial.println("Back or show lyrics");
        button_pressed("BACK_LYRICS");
        Serial.println(interFace);
      }
    }
  }


  // save the readinga. Next time through the loop, they'll be the lastButtonState:
  lastButtonState_rotaryUp = reading_Up;
  lastButtonState_rotaryDown = reading_Down;
  lastButtonState_Confirm = reading_Confirm;
  lastButtonState_Back_Lyrics = reading_back_Lyrics;
}


//Setup Function
void setup()
{
  //For the mp3 player
  mySoftwareSerial.begin(9600);
  Serial.begin(115200);


  if (!myDFPlayer.begin(mySoftwareSerial)) {  //Use softwareSerial to communicate with mp3.
    Serial.println(F("Unable to begin:"));
    Serial.println(F("1.Please recheck the connection!"));
    Serial.println(F("2.Please insert the SD card!"));
    while (true);
  }
  Serial.println(F("DFPlayer Mini online."));

  //For the LCD screen
  screen.init();
  screen.backlight();


  //Set serial communictaion time out 500ms
  myDFPlayer.setTimeOut(500);
  //----Set volume----
  myDFPlayer.volume(15);  //Set volume value (0~30).
  //----Set different EQ----
  myDFPlayer.EQ(DFPLAYER_EQ_NORMAL);
  //----Set device we use SD as default----
  myDFPlayer.outputDevice(DFPLAYER_DEVICE_SD);


  //Setting up the pins (buttons)
  pinMode(ROTARY_UP, INPUT_PULLUP);
  pinMode(ROTARY_DOWN, INPUT_PULLUP);
  pinMode(ROTARY_CONFRIM, INPUT_PULLUP);
  pinMode(BACK_LYRICS, INPUT_PULLUP);
  pinMode(XPIN, INPUT);
  pinMode(YPIN, INPUT);
  pinMode(ZPIN, INPUT);

  //Initial Display on the lcd for the menu
  screen.setCursor(4, 0);
  screen.print("Welcome");
  delay(1500);
  screen.clear();
  screen.setCursor(6, 0);
  screen.print("Menu");
  screen.setCursor(1, 1);
  screen.print("Unlocked Songs");

  //Set up the transceiver
  radio.begin();
  radio.openWritingPipe(addresses[1]); // 00002
  radio.openReadingPipe(1, addresses[0]); // 00001
  radio.setPALevel(RF24_PA_MIN);



  //Test ground
  //Serial.println(Unlocked_2010s_Movie[0]);
  //myDFPlayer.play(1);
  //read_information();
}

void loop()
{
  //Check if it is already 1 day after the last Jackpot. If so, add 1 chance to jackpot.
  if (millis() - Jackpot_Clock > OneDay) {
    accelerometer_time = 0;
    Jackpot_Chance += 1;
    Jackpot_Clock = millis();
  }

  //Check if it is already 1 day after the last reception of song. If so, allow reception again.
  if (millis() - Receive_Clock > OneDay) {
    song_received = false;
    Receive_Clock = millis();
  }


  //Update the interface according to the inputs
  read_accelerometer();
  read_buttons();

  //Check if there is song being shared to receive
  if (interFace == "Be Connected") {
    //Song_received is the bool variable that check whether a reception has already occured 
    if (interFace_Secondary == "Receive" && song_received == false) {
      if (interFace_Option == 1) {
        receive_song();
      }
    }
  }
}

Code for the “Dummy Arduino”:

//Library
#include <SPI.h>
#include <nRF24L01.h>
#include <RF24.h>
RF24 radio(8, 9); // CE, CSN

//Addresses for transceriver
const byte addresses[][6] = {"00001", "00002"};

//Variables for the button 
const int BUTTONPIN = 7;
int buttonState;             
int lastButtonState = HIGH; 

//For debouncing
unsigned long lastDebounceTime = 0;  
unsigned long debounceDelay = 50; 

//Change this to 1-5 for sharing songs. Any number for stimulating the competition.
int shared_song = 100000;

//Function to send a integer representing a song through transceiver. It can be used to send int for any purpose. 
void share_song() {
  radio.stopListening();
  radio.write(&shared_song, sizeof(shared_song));
  shared_song += 1;
  Serial.println("Shared");
}

void setup() {
  radio.begin();
  radio.openWritingPipe(addresses[0]); // 00002
  radio.openReadingPipe(1,addresses[1]); // 00001
  radio.setPALevel(RF24_PA_MIN);
  
  pinMode(BUTTONPIN, INPUT_PULLUP);
}


void loop() {
  /*
  int song = 1;
  radio.write(&song, sizeof(song));
  delay(1000);
  */

   int reading = digitalRead(BUTTONPIN);


  // If the switch changed, due to noise or pressing:
  if (reading != lastButtonState) {
    // reset the debouncing timer
    lastDebounceTime = millis();
  }

  if ((millis() - lastDebounceTime) > debounceDelay) {
    // whatever the reading is at, it's been there for longer than the debounce
    // delay, so take it as the actual current state:

    // if the button state has changed:
    if (reading != buttonState) {
      buttonState = reading;

      
      if (buttonState == LOW) {
        share_song();
      }
    }
  }
  lastButtonState = reading;
}

 

Invisibility in covid

I’d like to expand on my sound crit idea to incorporate the invisible factors around covid prevention. Three sensors I’d use – wind sensor, ultrasonic sensor, volume sensor. Together, they can provide the count on the number of people in a room, ventilation speed and how loud people are talking. This would feed information into a p5.js sketch that shows a visualization of what’s in a green, orange or red zone. This can also be accompanied by a sound when the red zone is reached, and potentially windows can be set to open automatically.
Alternatively, I could also work on a shopping mall entrance checker with a temperature sensor and a ml5.js sketch that recognizes whether the person in front of it is wearing a mask. If they’re not wearing a mask, the door doesn’t open to allow them to enter (probably use a servo to simulate this), even if their temperature is within range.

Are you set for the next couple hours?

Problem

With quarantine, I have not been leaving the house all that much, so I seem to have lost my intuition about temperature and my habit of always checking the weather in the morning. This has led me to be surprised by how hot/cold it gets if I’ve gone on a long walk or gone grocery shopping as well as when it starts raining before I’ve gotten home. This led me to the idea of a system that when I leave the house, would let me know about weather changes in the 2 following hours without bothering me. During initial discussion, Jet brought up the suggestion of having reminders only if the person isn’t prepared for the weather change as well as a notification if the person forgot a mask.

Solution

To realize my idea, I plan to make a system that would notify you depending on the weather change when you are leaving the house. It would be known if you are leaving the house rather than entering house because the motion sensor would not be triggered. When you open the door, the current weather would be compared to the weather in the next 2 hours and if it will rain, snow, and/or the temperature will change by 5°F or more you’ll be notified. This data would be collected by a python web scraper. For rain or snow, a pitter-patter type sound would be played by the speaker and temperature changes would be signified by the LED strip’s colors. To attract the person’s attention, this would flash the appropriate color of red-orange or blue three times before staying on for about 5 seconds more. These reminders would be noticeable but not a bother if you are already prepared. This is what I want to first accomplish and then will move on to implementing the RFID part as this is most important to me and it is easily usable by everyone and tags would not have to be put on clothes. Additionally, this would be my first time using WiFi on a project, connecting the Arduino to Python, and creating a web scraper, so I want to focus first on doing that well.

To create the system that would only notify the person if they aren’t prepared, I’d put an RFID scanner near the door frame beside the magnetic door switch so that the the tags on clothes could be scanned while you walk by. The same barcodes would be on things that fit the same purpose. For example, all masks would have one code, rain jacket and umbrella would have another, sweatshirts another, and light jackets another. This can be customized per user in the household code so that what they’d wear for certain conditions and the matching codes are associated with just that person. Each person would be differentiated by the RFID card that they carry on them whenever they leave the house, here the CMU ID card. For a full product using this, I’ve found laundry safe RFID tags that were pretty cheap and small so could work really well in this application.

Components I plan to use:

  • magnetic door switch
  • esp8266 module or board
  • speaker
  • led strip
  • motion sensor

Add ons

  • RFID sensor and cards

 

Pilates buddy

As a person who always sits in front of the monitor 24/7, my colleagues and I started to realize that we are beginning to establish another commonality other than our majors — back pain. Then I started to brainstorm around the problems of keeping up a good posture, and I got an idea of what other exercises require one to be in a straight back position.

Then I thought of pilates. Pilates, an exercise that requires one to not only have a straight back, but also a regulated breathing technique, and I thought making a pilates tracker would fit the agenda of the final crit.

There are two main interactions I am trying to tackle: the back posture and the breathing tempo.

For the back posture algorithm, I thought of manipulating the accelerometer’s XYZ values on each. If each value shifts in an inclined position, then the vibration motor 1 would buzz.

For breathing, two variations would be needed for the vibration: a short breath and a long breath. By having a button, one can change the tempo of the vibration motor 2. The vibration motor will pulsate in a standard breathing tempo.

Then the additional feature would be background music. It will be peaceful, zen music to corporate with one’s respiration as well. This will be done by integrating the mp3 function in the code.

 

The image on the left would be a more realistic appearance compared to the one on the right. The band will have the accelerometer in the middle, and the two vibration motors on each side.

 

This is what it would look like as a sketch.

Ambition:

  • Developing a UI
  • Visualizing the band with real scale components in them
  • Other components?

 

Interactive Restaurant Menu — Still in paper form!

Inspired by the LED bookmarks, I thought of how circuits could be integrated into traditionally paper like things. Restaurants menus have been evolving quite a lot, and I experience this culture shock every time I come back to China: they went from paper menu, to ordering from tablets/ipads, to having customers scanning a QR code to order and pay from their phone in the last few years. As much as I appreciate the convenience, I really enjoy the feeling of flipping through a paper menu.

LED bookmark

I understand many pros of having a digital ordering system for the customer end:

    • saves time for servers & the number of servers needed
    • avoid ordering mistakes
    • the order directly goes from customers to the kitchen without going through the servers
    • customers can see how much they will be spending right away

Adapting such system is costly, but the wide adoption indicates that the productivity beats the cost. However, in the US, we don’t really see restaurants that uses digital ordering system for the customers.

Some AR applications are used for restaurant menus, and there is a restaurant in London named Inamo that has interactive table tops. BUT I’m proposing a interactive menu that is still in the traditional paper form but takes advantage of the digital ordering system. 

Say when the customers presses on a dish on the paper menu, the dish he/she want to order lights up. The menu can display how much the customer will be paying. And when the customer finish ordering, he/she hits submit, and the order goes directly to the kitchen. The waiter can then collect the menu away, and the menu will be ready for the next customer to order.

As for components, I think I will need:

    • conductive tape
    • leds (either those flat/sewable leds or some fancier way of lighting things up while maintaining the overall paper like texture)
    • digit display/lcd to display price
    • a “submit” button
    • to select a dish, things that could work: tactile button, switch button, or even faking a button using tape

 

Class notes, 17 Nov 2020

Interactive jacket project from Baskinger’s Experimental Form class.

Look at the 2019 Final Crit for some good examples on elevator pitches and proposals.

Next Tuesday, 24 Nov, is the last day you can use A10.  We’ll make that a work day.  Thursday, 26 Nov, no class due to turkey day.

For Thursday, have a proposal posted to the blog’s Final Crit category.

 

Class notes, 12 Nov 2020

p5.js / arduino

Configure/test p5.js not required for final crit but lets make sure you can use it if you decide to.  The NYU ITP instructions are a good guide.  Do not use the online p5.js web editor, use local files and a local browser and text editor.

Final Crit

Start thinking about the final crit. We only have one more day to access A10, 24 Nov, and you should have your wish lists ready

For the final crit  combine visual, sound, and kinetics with a state machine and sensors to make something more accessible, usable, or uses universal design.

Here’s a rough schedule.  On some of the work days I will do a brief lecture based on questions or inspirational projects done in other physical computing projects.  The remainder of this time is making up for my not having open-office hours in A10.

  • 17 Nov — workshop proposals
  • 19 Nov — work day
  • 24 Nov — last day in A10
  • 26 Nov — vacation
  • 1 Dec — work day
  • 3 Dec — crit test flight
  • 8 Dec — work day
  • 10 Dec — final crit

Assignment for Tuesday

Crit warmup: come up with a problem to solve.  No need to write code (unless you want to), just a statement of the problem and your solution.

We will discuss these on Tuesday and brainstorm improvements.

Viewing assignment

These are based on the idea of how you can hide/obfuscate the man behind the curtain.

Former FBI Agent Explains How to Read Body Language

Former CIA Operative In Charge of Disguise Explains How Spies Use Disguises  (except for the model with a sleeve)

…and debunks spies and disguises in entertainment:

Don’t trip on what’s in front of you!

Problem:

People often trip over small objects that are in their path that they may not be expecting, especially if the object isn’t very tall. This is because the object is well below their sight line unless they are looking towards the ground. If someone’s on their phone, they are also more likely to trip over a small object. Blind people also of course have to be able to navigate around things that they cannot see. While they have a cane that will help them to know if something is in front of them by it hitting it, but sounds to notify something within a stride could be helpful.

Proposed Solution:

To help people with this problem, I propose a small wearable device strapped to the ankle that will buzz at different intervals based on how far an object is. This device is made up of a servo motor, ultrasonic distance sensor, accelerometer, and pancake vibration motor. The servo, distance sensor, and accelerometer are all attached. The servo moves such that the accelerometer is level so that the distance sensor is pointed straight ahead. This is important because as you walk, your leg is at various angles and doesn’t stay vertical. The buzzing from the vibration motor is a good way to know how far you are from something because the sound is pretty distinctive and has the added bonus of being able to feel it on your leg. For an actual version of this, I would package this smaller so that it could be actually something small attached to your leg that isn’t cumbersome.  This would go on both ankles so that you’d know if something was in your path for sure and not just on one side. I’d also use a bigger vibration motor so that the sound is a bit louder; with this, I could feel it and hear it fine, but if the house was louder, it would’ve been harder to hear. I also wish that I had soldered the leads of the vibration motor while I was in A10 because the wire kept coming loose while trying to take videos of me walking and I’d have to bend down to fix it and then when I stood up, it would come loose again; along the same vein, I wish I had been recording the whole time because a really good example while walking with the all different buzzing intervals happened when I forgot to restart it.

Proof of Concept:

Schematic
//changed smoothing example from https://www.arduino.cc/en/Tutorial/BuiltInExamples/Smoothing

#include <Servo.h>

Servo myLittleMotor;
const int servoPin = 11;
int servoPosition = 120;

#include <NewPing.h>
const int triggerPin = 8;
const int echoPin = 9;
int maxDistance = 400;
NewPing sonar(triggerPin, echoPin, maxDistance);
const int footSize = 25.4;//cm
int distance;

const int numReadings = 10;
int DISTreadings[numReadings];      // the readings from the analog input
int readIndex = 0;              // the index of the current reading
int DISTtotal = 0;                  // the running total
int DISTaverage = 0;                // the average

const int yPin = A4;
int yVal;

const int pancakePin = 6;

void setup() {
  myLittleMotor.attach(servoPin);


  pinMode(yPin, INPUT);


  pinMode(pancakePin, OUTPUT);

  Serial.begin(9600);
  myLittleMotor.write(servoPosition);
  for (int thisReading = 0; thisReading < numReadings; thisReading++) {
    DISTreadings[thisReading] = 0;
  }

}

void loop() {
  myLittleMotor.write(servoPosition);
  yVal = analogRead(yPin);

  distance = sonar.ping_cm();
  // subtract the last reading:
  DISTtotal = DISTtotal - DISTreadings[readIndex];
  // read from the sensor:
  DISTreadings[readIndex] = distance;
  // add the reading to the total:
  DISTtotal = DISTtotal + DISTreadings[readIndex];
  // advance to the next position in the array:
  readIndex = readIndex + 1;

  // if we're at the end of the array...
  if (readIndex >= numReadings) {
    // ...wrap around to the beginning:
    readIndex = 0;
  }

  // calculate the average:
  DISTaverage = DISTtotal / numReadings;
  // send it to the computer as ASCII digits
  //Serial.println(Yaverage);
  if (yVal > 352 && servoPosition > 0) {
    servoPosition -= 1;
  }
  else if (yVal < 348 && servoPosition < 180) {
    servoPosition += 1;
  }
  if (DISTaverage < (1.5 * footSize)) { //1 footSize past your foot
    static int start = millis();
    //Serial.println("closest");
    if ((millis() - start) < 200) {
      //digitalWrite(speakerPin,HIGH);
      digitalWrite(pancakePin, HIGH);
    }
    else if ((millis() - start) < 400) {
      digitalWrite(pancakePin, LOW);
    }
    else {
      start = millis();
    }
  }
  else if (DISTaverage < (1.5 * footSize + footSize)) {
    static int start2 = millis();
    if ((millis() - start2) < 200) {
      digitalWrite(pancakePin, HIGH);
    }
    else if ((millis() - start2) < 600) {
      digitalWrite(pancakePin, LOW);
    }
    else {
      start2 = millis();
    }
  }
  else if (DISTaverage < (1.5 * footSize + 2 * footSize)) {
    static int start3 = millis();
    if ((millis() - start3) < 200) {

      digitalWrite(pancakePin, HIGH);
    }
    else if ((millis() - start3) < 1000) {
      digitalWrite(pancakePin, LOW);
    }
    else {
      start3 = millis();
    }
  }
  else {
    //noTone(speakerPin);
    digitalWrite(pancakePin, LOW);
  }
  Serial.print(yVal);
  Serial.print('\t');
  Serial.println(DISTaverage);
  delay(25);
  //38.1
  //63.1
  //88.9
}