Sound in Physical Interactions

There is a music staircase in a mall in my hometown in China. I couldn’t find any videos for that mall, but here is something similar. Stepping on different stairs makes different notes.

 

This is one of the golf training aids that uses sounds to indicate something is wrong with your swing. This particular example is for if your swing is too long, but there are some other ones that monitor rotational angles and separation between upper and lower body.

This one does not have much interaction but I just wanted to put it here. Japanese singing toilets were very interesting…

Sound in Physical Interactions

  1. VR Art installation by Ray McClure

2. Microsoft Soundscape – Map delivered in 3D sound

Link for more information

3. Some interesting links:

Deep learning monitors human activity based on sound alone, able to parse whether a person in a room is sitting, standing, walking or falling.

Apple’s approach to harmonize sound with physical interactions by thinking of them as audio-haptic experiences

Use of Sound in Physical Interaction

Akousmaflore: The Sound of Plant

The interaction installation is composed of plants that react to gentle contact by producing a computer-synthesized sound. The texture and volume of the sound depends on movements and gestures made by the person.

Taiko no  Tatsujin

This is a arcade/video game that based on music and rhyme. Players can recognize if they are doing a good job by listening to the rhyme of the music, the rhyme made by the drum, and feedback lines like “50 strikes”.

Augmented Drilling Machine

The device makes pulsing sound of different pitches if the drill is deviated from the ideal angle to different angles.

Make it So Chapter 6,10

After reading Make it So Chapters 6 and 10, I have realized that sound has many more roles than I thought. To people, sound can provide clues of the size of the space, direction, or even the distance. It is far more capable of being used as an alert/alarm. The sound is one of the strongest elements to involve in a device. It does not only interacts with people physically but also psychologically. Sound is a valuable part of human memory. This means, that in a device, one can even approach to condition one to the sound of the device. For example, when we hear our phone ding, the thumb is already tapping the screen for thumbprint recognition/password. The power of sound can also be seen in emotional interaction. For example, music can make one feel the whole spectrum of emotions such as being gloomy or calm. This can be used in devices that cooperate for one’s emotional support in situations like when one indicates physical signs of being angry such as faster heartbeat/increase in blood pressure, the device can detect those signs and play calm music. Sound can also interact with the user from sound to sound —  like a conversation. As Chapter 10 mentions, features like playback can also play a big role in devices even if the sounds don’t necessarily make a full-on conversation (even siri today fails to make a smooth conversation!) Sound gives people a lot of space to work with, and the fact that sound doesn’t necessarily have to be audible to human sounds makes it even more potent.  (If anyone’s a fan of the netflix show Umbrella Academy, *spoiler alert* I think you would know that the sound is so powerful)

Sound in Physical Interaction Examples

Korean rice cooker: when the rice is ready, the voice says “Delicious rice is ready to be served” and the lid automatically opens. Based on time/water level one sets, the rice cooker serves the most ideal type of rice.

Theft alarm in stores: When people walk out with a product without paying, the alarm detects it and makes a loud noise.

Automatic Cat feeder at my house: whenever it is user’s set time, the voice recording of me saying “time to eat kitties!”(which I trained my cats to come and eat when I say this) plays automatically, and the chamber inside the feeder rotates and sends out cat food.

Not related to this topic, but I also made something similar in my last semester’s class, which is an automatic feeder that detects cats movement and also a cat exerciser.

Sound in Physical Interaction

Secret Knock Detecting Door Lock:

This device can recognize a secret knock pattern that’s up to 20 knocks long and unlock the door if it is correct! Instructions on how to make it can be found here.

The world’s deepest bin

This trash can makes it sound like garbage thrown in is taking a really long time. It made throwing trash away more fun and even got some people to pick up litter.

Urban Lights Contacts:

This interactive sound installation changes the sound output based on electrostatic connections; in other words, it will make varying sounds from how close people are and if people are touching. This is a really cool concept, but definitely something that would give me anxiety to try because of COVID.

Bring the noise + reading assignment

Use your Arduino, speaker, and servos or solenoids to make some noise.  Post something to the blog if you like, this is mostly an exercise making sure you have all the hardware we need for the Sound section.

Reading assignment: Make It So chapters 6 and 10.

Post something to Looking Outward showing use of sound in physical interaction.

Arduino source formatting tool

I’ve added a tool that shows up in the Post editor as the icon “{…}”.   To post the code below, I clicked on that icon, pasted in my code, turned off indentation, then selected C++, and hit “OK”.

 

// -*-c++-*-
/*
  The MIT License (MIT)

  Copyright (c) 2017 J. Eric Townsend

  Permission is hereby granted, free of charge, to any person obtaining a copy
  of this software and associated documentation files (the "Software"), to deal
  in the Software without restriction, including without limitation the rights
  to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
  copies of the Software, and to permit persons to whom the Software is
  furnished to do so, subject to the following conditions:

  The above copyright notice and this permission notice shall be included in
  all copies or substantial portions of the Software.

  THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
  IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
  FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
  AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
  LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
  OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
  THE SOFTWARE.
*/

/*
  using interrupts to stop/alter a process that's eating up the loop()
cycle with delays

 */

static const int potOnPin = A1;
static const int potOffPin = A1;


static const int togglePin = 2;

static const int LED1Pin = 5;
static const int LED2Pin = 6;
static const int LED3Pin = 9;

int LEDOnDuration = 1000;  // milliseconds
int LEDOffDuration = 1000;  // milliseconds
long LEDOnTime = 0;
long LEDOffTime = 0;;
bool LEDState = false;
bool LEDStatePrev = false;
int toggleState = 0;

bool flagState = false;
const bool isInterrupt = true;

// TODO: how do we stop the for() loops so we can do another analogRead()?
void SwitchPressed()
{
    // we can set the delay to 0 and fall out of the loops
  //LEDOnDuration = 0;
  //LEDOffDuration = 0;
    // or we can modify a flag variable and use that to exit the loop
       flagState = true;
}

void setup() {
  Serial.begin(115200);

  pinMode(potOnPin, INPUT);
  pinMode(potOffPin, INPUT);
  pinMode(togglePin, INPUT);

  pinMode(LED1Pin, OUTPUT);
  pinMode(LED2Pin, OUTPUT);
  pinMode(LED3Pin, OUTPUT);

  if (isInterrupt) {
    attachInterrupt (digitalPinToInterrupt(togglePin), SwitchPressed, CHANGE);
  }
}

void loop() {

  if (!isInterrupt) {
    flagState = digitalRead(togglePin);
  }

  if (flagState) {
    Serial.println("flagState true");
  }

  // take input from the pots to determine on/off times
  int pot = analogRead(potOnPin);
  if (pot != LEDOnDuration) {
   //Serial.print("on: ");
    //Serial.println(pot);
    LEDOnDuration = map(pot, 0, 1024, 0, 255);
  }
  pot = analogRead(potOffPin);
  if (pot != LEDOffDuration) {
   // Serial.print("off: ");
   // Serial.println(pot);
    LEDOffDuration = map(pot, 0, 1024, 0, 255);
  }


  // now let's write some really bad code
  // turn the LEDs on
  for (int i = 0; i < 255; i++) {
      if (flagState) {
        break;
  // is more reliable than setting i = 256;
          // if we set flagState back to false here we
          // won't make a change in the next loop where we
          // turn LEDs off
      }
      else {
          analogWrite(LED1Pin, i);
          analogWrite(LED2Pin, i);
          analogWrite(LED3Pin, i);
          delay(LEDOnDuration);
      }
  }
  // turn the LEDs off
  for (int i = 255; i > 0; i--) {
      if (flagState) {
          break;
  // is more reliable than setting i = 256;
          // if we set flagState back to false here we
          // won't make a change in the next loop where we
          // turn LEDs on
      }
      else {
          analogWrite(LED1Pin, i);
          analogWrite(LED2Pin, i);
          analogWrite(LED3Pin, i);
          delay(LEDOffDuration);
      }
  }

  // this is the proper place to turn off the flagState.
  // We could do this with just a statement and never check
  // its status, but using a test makes debugging easier
  if (flagState == true) {
      flagState = false;
      // because we could add this
      Serial.println("flagState was on, now turned off");
  }

}

 

Braille Translator

Problem

It can be very difficult for people with seeing difficulty to orient themselves. In fact, it is already difficult enough to locate the braille signs, which is supposed to help them. In addition, there are situations where visual clues are necessary, such as deciding which elevator goes up when two arrive at the same time. Here is a video that mentions such difficulty.

As a result, people with visual difficulty have to memorize routes. It is also very challenging for them to travel through a new district or locate a room in an unfamiliar building.

Solution

The key that causes this problem is the lost of visual clues. When we walk into a new building, we can look for signs for directions. When we travel through a new place, we can use google map as a guide.  However, for people with seeing difficulty, the same clues are either impossible to get, like the road sings for example, or very inaccessible like the braille sign that hides behind a door.

To solve this problem, I would like to translate the visual clues into braille on a device that is accessible. In other words, instead of looking for physical sign of braille, the user can read the signs in braille on a device in his or her hand. This device, however, can not only be used as a navigator but also a “seeing” tool to read the visuals in a space or environment.

The design of the prototype

As shown in the figure above, the device have 12 digits for displaying letters and punctuations. Each digit is made of 6 solenoids and will display alphabet in braille. The rotary encode on the right is for flipping to the next 12 letters in the sentences. There are also a confirm button and a back button. This device can be connected to smart phone and indoor devices through Bluetooth or wi-fi.

Suppose a person with difficulty wants to go to new place, he or she can use app on  his or her smartphone to determine the route. The person can then use the device to navigate while walking. When there are road signs or alerts that need attention, the device will vibrate and display the message. The device will go back to the route when the alert was read and confirm button pressed. The person can also use the back button to check the last direction or instruction.

Of course, the effectiveness of this device depends heavily on how small we can make it to be. With the current material of mini-solenoid, I can make it to have a length of 18 cm and a width of 7.2 cm. This cannot work effective as a navigation tool because it is too big and cumbersome. However, it can still be used while stationary such as in a restaurant, museum, and a room.

For future design

If we have the interactive surface technology in the future, this is the design I will give to the device. It wrap around the arm of the user so that the user can hold a white cane in one hand, and read the information using another hand. No more buttons and rotary encoder are needed because everything can be displayed and interacted with through the surface.

Proof of Logic

The components of the demonstration

The demo is made up of a solenoid, a Bluetooth module, 6 LED, 4 buttons, and a vibration motor. The LEDs represent the 6-dot grid of the braille. The corresponding grid is shown below.

LED braille grid

Because the demo-device is relatively complex, it will be explained with video of demonstration.

This shows the normal function of the solenoid. In the later demonstration it does not function correctly due to the insufficient current arduino UNO supplies.

The two button pressed in the video represents the rotary encode that allows the user to flip to the next or previous 12 letters.

The buttons pressed in the video are confirm button and back button, which allows the user to display the next or previous message instruction.

Finally, in the video alerts are sent from the PC to the device through Bluetooth. The device then vibrate and display the alert with the corresponding braille letter.

Schematic:

Code:

//Defining the pins
#define solenoide 12
#define motor 13
#define rotary_up 5
#define rotary_down 4
#define confirm 3
#define back 2
#define LED1 11
#define LED2 10
#define LED3 9
#define LED4 8
#define LED5 7
#define LED6 6

//Variables
//constant variables
char *INPUT_STRINGS[] = {"information destk is at first floor","elevator is to the right by ten meters" ,"this is room b ten"};

//Message is like the route and direction
int message;
int message_length;
String string;
int start_letter;

//Alert is like the signs or alerts on the street or in buildings
String incoming_alert; //string that stores the incoming message
String alert[5];
int alert_index;
bool has_alert;
bool vibration_on;

//variable for interrupt, note that message is also a ISR variable
const bool isInterrupt = true;
int message_number = 3;

//timer
//Clock 1 is the timer for checking the rotary encoder
unsigned long clock1 = 0; // variable for timing
const int INTERVAL1 = 300; // milliseconds between updates

//Clock 2 is for serial printing for illustration
unsigned long clock2 = 0; // variable for timing
const int INTERVAL2 = 2000; // milliseconds between updates

//Clock 3 is for serial printing for illustration
unsigned long clock3 = 0; // variable for timing
const int INTERVAL3 = 1500; // milliseconds between updates


// the interrupt method
// NOTE:  we shouldn't use Serial.prinln(), delay(), and many
// other functions in an interrupt method
void ConfirmSwitchPressed()
{
  if (digitalRead(confirm) == LOW) {
    if (has_alert == true) {
      //alert index also show how many alerts are in the alert array
      if (alert_index == 1) {
        has_alert = false;
      }

      for (int i; i < alert_index; i++) {
        alert[i] = alert[i + 1];
      }

      if (alert_index != 0) {
        alert_index -= 1;
      }

      Serial.println("Alert read");
      return;
    }


    if (has_alert == false) {
      message == message_number ? message = 0 : message++;
      start_letter = 0;

      Serial.println("Next Message");
    }
  }
}

void BackSwitchPressed()
{
  if (digitalRead(back) == LOW) {

    if (has_alert == false) {
      message == 0 ? message = message_number : message--;
      start_letter = 0;

      Serial.println("Previous Message");
    }
  }
}

//Functions
//demonstration function that shows how to move solenoid
void move_solenoid() {
  digitalWrite(solenoide, HIGH);
  delay(1000);
  digitalWrite(solenoide, LOW);
  delay(1000);
}

//Demonstration function
void vibrate_motor() {
  digitalWrite(motor, HIGH);
  delay(2000);
  digitalWrite(motor, LOW);
  delay(2000);
}


void show_letters() {
  if (has_alert == true) {
    braille(alert[0][start_letter]);
  }

  if (has_alert == false) {
    braille(INPUT_STRINGS[message][start_letter]);
  }

  //if there are actually 12 digits, each made of 6 solenoid
  //int digit;
  //for (i=start_letter, i< start_letter +13,i++) {
  //     braille(digit, INPUT_STRINGS[message][start_letter + i])
  //     digit += 1;
  //    }
}

//This function put the incoming alert into the alert array
void insert_alert() {
  //-1 indicates that there is no \ in the alert message
  int null_index = incoming_alert.indexOf('.');
  if (null_index != -1 and alert_index < 4) {
    alert[alert_index] = incoming_alert.substring(0, null_index);
    alert_index += 1;
    //Tell the system there are alerts
    has_alert = true;

    //Turn the vibration and its corresponding timer on
    digitalWrite(motor, HIGH);
    vibration_on = true;
    clock3 = millis() + INTERVAL3;

    //Clear the string for the next incoming alert
    incoming_alert = "";

    Serial.print("Alert received, ");
    Serial.println("vibration on.");
  }
}



//In the actual desgin instead of this demo, the 6 LEDs are 6 solenoides
void braille(char input) {
  digitalWrite(LED1, LOW);
  digitalWrite(LED2, LOW);
  digitalWrite(LED3, LOW);
  digitalWrite(LED4, LOW);
  digitalWrite(LED5, LOW);
  digitalWrite(LED6, LOW);
  digitalWrite(solenoide, LOW);


  if (input == 'a') {
    digitalWrite(LED1, HIGH);

    digitalWrite(solenoide, HIGH);
  }

  if (input == 'b') {
    digitalWrite(LED1, HIGH);
    digitalWrite(LED2, HIGH);

    digitalWrite(solenoide, HIGH);
  }

  if (input == 'c') {
    digitalWrite(LED1, HIGH);
    digitalWrite(LED4, HIGH);

    digitalWrite(solenoide, HIGH);
  }

  if (input == 'd') {
    digitalWrite(LED1, HIGH);
    digitalWrite(LED4, HIGH);
    digitalWrite(LED5, HIGH);

    digitalWrite(solenoide, HIGH);
  }

  if (input == 'e') {
    digitalWrite(LED1, HIGH);
    digitalWrite(LED5, HIGH);

    digitalWrite(solenoide, HIGH);
  }

  if (input == 'f') {
    digitalWrite(LED1, HIGH);
    digitalWrite(LED2, HIGH);
    digitalWrite(LED5, HIGH);

    digitalWrite(solenoide, HIGH);
  }

  if (input == 'g') {
    digitalWrite(LED1, HIGH);
    digitalWrite(LED2, HIGH);
    digitalWrite(LED4, HIGH);
    digitalWrite(LED5, HIGH);

    digitalWrite(solenoide, HIGH);
  }

  if (input == 'h') {
    digitalWrite(LED1, HIGH);
    digitalWrite(LED2, HIGH);
    digitalWrite(LED5, HIGH);

    digitalWrite(solenoide, HIGH);
  }

  if (input == 'i') {

    digitalWrite(LED2, HIGH);
    digitalWrite(LED4, HIGH);

  }

  if (input == 'j') {
    digitalWrite(LED2, HIGH);
    digitalWrite(LED4, HIGH);
    digitalWrite(LED5, HIGH);
  }

  if (input == 'k') {
    digitalWrite(LED1, HIGH);
    digitalWrite(LED3, HIGH);

    digitalWrite(solenoide, HIGH);

  }

  if (input == 'l') {
    digitalWrite(LED1, HIGH);
    digitalWrite(LED2, HIGH);
    digitalWrite(LED3, HIGH);

    digitalWrite(solenoide, HIGH);
  }

  if (input == 'm') {
    digitalWrite(LED1, HIGH);
    digitalWrite(LED3, HIGH);
    digitalWrite(LED4, HIGH);

    digitalWrite(solenoide, HIGH);
  }

  if (input == 'n') {
    digitalWrite(LED1, HIGH);
    digitalWrite(LED5, HIGH);
    digitalWrite(LED3, HIGH);
    digitalWrite(LED4, HIGH);

    digitalWrite(solenoide, HIGH);
  }

  if (input == 'o') {
    digitalWrite(LED1, HIGH);
    digitalWrite(LED5, HIGH);
    digitalWrite(LED3, HIGH);

    digitalWrite(solenoide, HIGH);
  }

  if (input == 'p') {
    digitalWrite(LED1, HIGH);
    digitalWrite(LED2, HIGH);
    digitalWrite(LED3, HIGH);
    digitalWrite(LED4, HIGH);

    digitalWrite(solenoide, HIGH);
  }

  if (input == 'q') {
    digitalWrite(LED1, HIGH);
    digitalWrite(LED2, HIGH);
    digitalWrite(LED3, HIGH);
    digitalWrite(LED4, HIGH);
    digitalWrite(LED5, HIGH);

    digitalWrite(solenoide, HIGH);
  }


  if (input == 'r') {
    digitalWrite(LED1, HIGH);
    digitalWrite(LED2, HIGH);
    digitalWrite(LED3, HIGH);
    digitalWrite(LED5, HIGH);

    digitalWrite(solenoide, HIGH);
  }



  if (input == 's') {
    digitalWrite(LED2, HIGH);
    digitalWrite(LED3, HIGH);
    digitalWrite(LED4, HIGH);

  }


  if (input == 't') {
    digitalWrite(LED2, HIGH);
    digitalWrite(LED3, HIGH);
    digitalWrite(LED4, HIGH);
    digitalWrite(LED5, HIGH);
  }


  if (input == 'u') {
    digitalWrite(LED1, HIGH);
    digitalWrite(LED3, HIGH);
    digitalWrite(LED6, HIGH);

    digitalWrite(solenoide, HIGH);
  }


  if (input == 'v') {
    digitalWrite(LED1, HIGH);
    digitalWrite(LED2, HIGH);
    digitalWrite(LED3, HIGH);
    digitalWrite(LED6, HIGH);

    digitalWrite(solenoide, HIGH);

  }


  if (input == 'w') {

    digitalWrite(LED2, HIGH);
    digitalWrite(LED6, HIGH);
    digitalWrite(LED4, HIGH);
    digitalWrite(LED5, HIGH);
  }


  if (input == 'x') {
    digitalWrite(LED1, HIGH);
    digitalWrite(LED3, HIGH);
    digitalWrite(LED4, HIGH);
    digitalWrite(LED6, HIGH);

    digitalWrite(solenoide, HIGH);
  }



  if (input == 'y') {
    digitalWrite(LED1, HIGH);
    digitalWrite(LED6, HIGH);
    digitalWrite(LED3, HIGH);
    digitalWrite(LED4, HIGH);
    digitalWrite(LED5, HIGH);

    digitalWrite(solenoide, HIGH);
  }


  if (input == 'z') {
    digitalWrite(LED1, HIGH);
    digitalWrite(LED6, HIGH);
    digitalWrite(LED3, HIGH);
    digitalWrite(LED5, HIGH);

    digitalWrite(solenoide, HIGH);
  }


  if (input == ',') {
    digitalWrite(LED2, HIGH);
  }


  if (input == '.') {
    digitalWrite(LED2, HIGH);
    digitalWrite(LED6, HIGH);
    digitalWrite(LED5, HIGH);
  }

  if (input == '!') {
    digitalWrite(LED2, HIGH);
    digitalWrite(LED3, HIGH);
    digitalWrite(LED5, HIGH);
  }
}



void setup() {
  // Setup the pin modes
  pinMode(solenoide, OUTPUT);
  pinMode(motor, OUTPUT);
  pinMode(LED1, OUTPUT);
  pinMode(LED2, OUTPUT);
  pinMode(LED3, OUTPUT);
  pinMode(LED4, OUTPUT);
  pinMode(LED5, OUTPUT);
  pinMode(LED6, OUTPUT);


  //When the buttons are pushed, digitalRead shows 0
  pinMode(rotary_up, INPUT_PULLUP);
  pinMode(rotary_down, INPUT_PULLUP);
  pinMode(confirm, INPUT_PULLUP);
  pinMode(back, INPUT_PULLUP);


  //Initiate the serial monitor
  Serial.begin(9600);

  // attach the interrupt pin to a method
  if (isInterrupt) {
    attachInterrupt (digitalPinToInterrupt(confirm), ConfirmSwitchPressed, FALLING);
    attachInterrupt (digitalPinToInterrupt(back), BackSwitchPressed, FALLING);
  }


}


void loop() {
  //Determining the number of messages/alerts and the length of each message

  if (has_alert == true) {
    string = alert[0];
    message_length = string.length();
  }

  if (has_alert == false) {
    string = INPUT_STRINGS[message];
    message_length = string.length();
  }

  //Receiving the characters of incoming alerts
  while (Serial.available()) {
    //while there is data available on the serial monitor
    //Serial.println(Serial.read());
    incoming_alert += char(Serial.read()); //store string from serial command
  }

  insert_alert();

  //Check if any movement has been made on the rotary encoder
  if (millis() >=  clock1) {

    if (digitalRead(rotary_up) == LOW) {
      if (start_letter + 12 <= message_length) {
        start_letter += 12;
        Serial.println("Next 12 letters");
      }
    }

    if (digitalRead(rotary_down) == LOW) {
      if (start_letter - 12 >= 0)  {
        start_letter -= 12;
        Serial.println("Last 12 letters");
      }
    }

    clock1 = millis() + INTERVAL1 ;

  }

  show_letters();

  //For serial printing of demonstration
  if (millis() >=  clock2) {

    if (incoming_alert != "") {
      Serial.print("Incoming Alert: ");
      Serial.println(incoming_alert);
    }

    //Serial.print("The current message or alert: ");
    Serial.println(string);

    
    clock2 = millis() + INTERVAL2 ;

  }

  //For 2-second duration of the vibration motor
  if (millis() >=  clock3 and vibration_on == true) {
    digitalWrite(motor, LOW);
    vibration_on = false;
    Serial.println("vibration off.");
  }


}

 

Worry free curtains and an alarm clock supplement that’s natural!

Problem:

Do you like having natural light in the room during the day, but often forget to close the curtains at night and so get woken up earlier than you wanted in the morning? Do you ever just want to wake up with natural light in the room, but not get woken up too early by it? After not getting enough sleep, do you find yourself having trouble getting up to your alarm or being blinded when you turn on the light or open the curtains because your eyes aren’t used to it? If you’re like me and have answered yes to these questions, this worry free curtain and alarm clock supplement could be the solution!

Proposed Solution:

Enter the time times you want to wake up each day of the week, the current date and time, upload the program, and that’s it your curtains will open according to the time you want to wake up and when you went to bed (set by pressing button) and close when it’s evening and dark. If you didn’t get nearly enough sleep, a fan will also blow in your face when your alarm goes off to help ensure you wake up. The fan could be an overhead one, but one on a nightstand or a taller one on the floor next to the bed would be best. I used the left and right threaded rods coupled together that I had from a previous project to be able to control the open and closing of the curtains movement. The logic I used for when the curtains should open compared to when you want to wake up and when you went to sleep were set by my preferences wherein the less sleep I got, the earlier the curtains would open based on time frames of amount of sleep. For me if I’ve gotten less sleep, I wake up to slight changes less than if I got enough sleep. Also, the fan is set to blow at a low speed for a minute when you’ve had under 5 hours sleep and high speed for 30 seconds if you’ve had between 5 and 6 hours. This is because higher speed will be more annoying and so needs to be on less; the lower speed is for the least sleep case because you are more likely to get sick on less sleep. I used a stepper motor to control the opening and closing of the curtains as that is what I had available and I can easily control the distance moved forward and reverse precisely. I would’ve used a faster motor if available and ideally a DC motor and an encoder instead as I’d be able to control the amounts forward and reverse easily and it’s more efficient in this scenario. Additionally, if I had a pressure sensor, the time you went to bed would be detected by the time you went on the bed and didn’t get up again before the alarm. This way you wouldn’t have to remember to click a button. To add to this, I would add a speaker so that this could be used for an alarm sound itself too. Finally, as someone mentioned during the crit, an app would be really useful for inputting the parameters and quickly making any changes. With the materials I was able to use and the time given, I’m happy with this prototype and in the future may make this for myself on an actual scale.

For reference, this is what the curtains in my room look like:

Ideal fans:

small fan for nightstand
floor stand fan to be put near  head of bed

Proof of Concept:

Closing the curtains after 6:00PM and dark

Setting the time went to bed, opening curtains, and blowing the fan.

You might notice that in this video, the printing format changed from day, month, year to month, day, year. This was an intentional change after the first video because it’s the format we are accustomed to.

Circuit
#include <AccelStepper.h>

// stepper Motor pin definitions:
#define motorPin1  9      // IN1 on the ULN2003 driver
#define motorPin2  10      // IN2 on the ULN2003 driver
#define motorPin3  11     // IN3 on the ULN2003 driver
#define motorPin4  12     // IN4 on the ULN2003 driver

// Define the AccelStepper interface type; 4 wire motor in half step mode:
#define MotorInterfaceType 8

// Initialize with pin sequence IN1-IN3-IN2-IN4 for using the AccelStepper library with 28BYJ-48 stepper motor:
AccelStepper stepper = AccelStepper(MotorInterfaceType, motorPin1, motorPin3, motorPin2, motorPin4);
//above from  https://www.makerguides.com to learn how to use this specific stepper motor and driver

#include <Wire.h>
#include <RTC.h>
const int photoPin = A5;
static DS3231 RTC;

//input times for each day with hr then min; 24 hr time but as it'll be morning, it shouldn't make too much of a difference
int wakeUpTimes[7][2] ={{12, 12}, //sun
                        {9, 0}, //mon
                        {9, 35}, //tues
                        {8, 55}, //wed
                        {9, 30}, //thurs
                        {10, 10}, //fri
                        {11, 11}};//sat

const int fanMotorPin = 3;
int low = 50;
int high = 150;
int fanSpeed;
int turnOnFan;
int fanDuration;

int timeSlept[3];
int openCurtainsXbeforeWakeUpTime;
int sleep = 0;
const int buttonPin = 2;

int timeWentToBed[3];

void calcTimeSlept() {
  //seconds
  int secs = 60 - timeWentToBed[0];
  if (secs == 60) {
    timeSlept[2] = 0;
  }
  else {
    timeSlept[2] = secs;
  }

  //mins
  if (wakeUpTimes[RTC.getWeek() - 1][1] >= timeWentToBed[1]) {
    if (timeSlept[2] == 0) {
      timeSlept[1] = wakeUpTimes[RTC.getWeek() - 1][1] - timeWentToBed[1];
    }
    else {
      timeSlept[1] = wakeUpTimes[RTC.getWeek() - 1][1] - timeWentToBed[1] - 1;
    }
  }
  else {
    if (timeSlept[2] == 0) {
      timeSlept[1] = wakeUpTimes[RTC.getWeek() - 1][1] + 60 - timeWentToBed[1];
    }
    else {
      timeSlept[1] = wakeUpTimes[RTC.getWeek() - 1][1] + 60 - timeWentToBed[1] - 1;
    }
  }

  //hrs
  if (timeWentToBed[0] < 12) {
    if (wakeUpTimes[RTC.getWeek() - 1][1] >= timeWentToBed[1]) {
      timeSlept[0] = wakeUpTimes[RTC.getWeek() - 1][0] - timeWentToBed[0];
    }
    else {
      timeSlept[0] = wakeUpTimes[RTC.getWeek() - 1][0] - timeWentToBed[0] - 1;
    }
  }
  else { //if went to bed before midnight, need to add hours bc subtraction not right otherwise
    int hrsBeforeMidnight = 24 - timeWentToBed[0];
    if (wakeUpTimes[RTC.getWeek() - 1][1] >= timeWentToBed[1]) {
      timeSlept[0] = wakeUpTimes[RTC.getWeek() - 1][0] + hrsBeforeMidnight;
    }
    else {
      timeSlept[0] = wakeUpTimes[RTC.getWeek() - 1][0] + hrsBeforeMidnight - 1;
    }
  }
}

void goinToSleep() {
  if (digitalRead(buttonPin) == HIGH) {
    sleep = 1;
  }
}

void openCurtains() {
  while (stepper.currentPosition() != 4096 * 7) { //choosing this bc it was enough rev to at least show some opening
    stepper.setSpeed(1000);
    stepper.runSpeed();
  }
}
void closeCurtains() {
  while (stepper.currentPosition() != 0) {
    stepper.setSpeed(-1000);
    stepper.runSpeed();
  }
}

void setup() {
  // put your setup code here, to run once:
  stepper.setMaxSpeed(1000);
  Serial.begin(9600);
  RTC.begin();

  RTC.setDay(16);
  RTC.setMonth(10);
  RTC.setYear(2020);

  RTC.setWeek(6);

//to show setting time went to bed from button for Fri
  RTC.setHours(10);
  RTC.setMinutes(8);
  RTC.setSeconds(0);
  
//to show fan turning on when not enough sleep for Mon
//  RTC.setHours(8);
//  RTC.setMinutes(59);
//  RTC.setSeconds(30);

////to show closing:
//  stepper.setCurrentPosition(4096 * 7);
  
  //to show curtains closing past 5 and dark
//  RTC.setHours(17);
//  RTC.setMinutes(59);
//  RTC.setSeconds(45);
  
//to show opening:
    stepper.setCurrentPosition(0);
  //to show curtain opening:
//  RTC.setHours(8);
//  RTC.setMinutes(49);
//  RTC.setSeconds(58);

  RTC.setHourMode(CLOCK_H24);

  pinMode(fanMotorPin, OUTPUT);

  attachInterrupt (digitalPinToInterrupt (buttonPin), goinToSleep, HIGH);

  /*set time went to bed to hr before wakeup as default after woken up
        will change if/when press button before going to sleep
        this way if you forget still get this, and assuming if didn't press,
        you went to bed really late
  */
  //commenting out for demo of button to set time went to bed
  timeWentToBed[0] = wakeUpTimes[RTC.getWeek() - 1][0] - 1;
  timeWentToBed[1] = wakeUpTimes[RTC.getWeek() - 1][1];
  timeWentToBed[2] = 0;
}

void loop() {
  int  photoVal = analogRead(photoPin);
  //Serial.println(photoVal);
  if (RTC.getHours() >= 18) {//6:00PM
    if (photoVal < 600) {
      //if it's past 6PM and dark(here just chose 600, but would be lower in real thing), close curtains
      closeCurtains();
    }
  }
  if (sleep == 1) {
    Serial.println("clicked");
    timeWentToBed[0] = RTC.getHours();
    timeWentToBed[1] = RTC.getMinutes();
    timeWentToBed[2] = RTC.getSeconds();
    calcTimeSlept();
    sleep = 0;
  }
  if (timeSlept[0] >= 8) {
    openCurtainsXbeforeWakeUpTime = 5; //mins
  }
  else if (6 <= timeSlept[0] && timeSlept[0] < 8) {
    openCurtainsXbeforeWakeUpTime = 30; //mins
  }
  else if (5 <= timeSlept[0] && timeSlept[0] < 6) {
    openCurtainsXbeforeWakeUpTime = 45; //mins
    turnOnFan = 1;
    fanSpeed = high;
    fanDuration = 30;
  }
  else {//under 5 hrs
    //for demo
    //openCurtainsXbeforeWakeUpTime = 1; //mins
    openCurtainsXbeforeWakeUpTime = 55; //mins
    turnOnFan = 1;
    fanSpeed = low;
    fanDuration = 60;
  }
  if (wakeUpTimes[RTC.getWeek() - 1][1] >= openCurtainsXbeforeWakeUpTime) {
    if (RTC.getHours() == wakeUpTimes[RTC.getWeek() - 1][0]) {
//Serial.println(openCurtainsXbeforeWakeUpTime);
        //Serial.println((wakeUpTimes[RTC.getWeek() - 1][1] - openCurtainsXbeforeWakeUpTime));
      if (RTC.getMinutes() == (wakeUpTimes[RTC.getWeek() - 1][1] - openCurtainsXbeforeWakeUpTime)) {
        openCurtains();
        timeWentToBed[0] = wakeUpTimes[RTC.getWeek() - 1][0] - 1;
        timeWentToBed[1] = wakeUpTimes[RTC.getWeek() - 1][1];
        timeWentToBed[2] = 0;
      }
    }
  }
  else {//minutes of wakeUpTime<time to subtract to wake up
    if (RTC.getHours() == (wakeUpTimes[RTC.getWeek() - 1][0] - 1)) { //took hr off here
      //right side of eq subtracted amount of time left after changing hr
      if (RTC.getMinutes() == (60 - openCurtainsXbeforeWakeUpTime + wakeUpTimes[RTC.getWeek() - 1][1])) {
        openCurtains();
        timeWentToBed[0] = wakeUpTimes[RTC.getWeek() - 1][0] - 1;
        timeWentToBed[1] = wakeUpTimes[RTC.getWeek() - 1][1];
        timeWentToBed[2] = 0;
      }
    }
  }
  
  if (RTC.getHours() == wakeUpTimes[RTC.getWeek() - 1][0]) {
    if (RTC.getMinutes() == wakeUpTimes[RTC.getWeek() - 1][1]) {
      //Serial.println(RTC.getSeconds());
      if (turnOnFan) {
        if (RTC.getSeconds() <= fanDuration) {
          //Serial.println(RTC.getSeconds());
          analogWrite(fanMotorPin, fanSpeed);
        }
        else {
                    analogWrite(fanMotorPin, 0);
          turnOnFan = 0;        
          }
      }

    }
  }
  static int startPrint=millis();
  if ((millis()-startPrint)>=1000){//putting this here so don't use delay while also showing the times
   switch (RTC.getWeek())
    {
      case 1:
        Serial.print("SUN");
        break;
      case 2:
        Serial.print("MON");
        break;
      case 3:
        Serial.print("TUE");
        break;
      case 4:
        Serial.print("WED");
        break;
      case 5:
        Serial.print("THU");
        break;
      case 6:
        Serial.print("FRI");
        break;
      case 7:
        Serial.print("SAT");
        break;
    }
    Serial.print(" ");
    Serial.print(RTC.getMonth());
    Serial.print("-");
    Serial.print(RTC.getDay());
    Serial.print("-");
    Serial.print(RTC.getYear());
  
    Serial.print(" ");
  
    Serial.print(RTC.getHours());
    Serial.print(":");
    Serial.print(RTC.getMinutes());
    Serial.print(":");
    Serial.println(RTC.getSeconds());

    startPrint=millis();
  }
  //  delay(1000);
}