Category: Assignments

Chapter 3:

Throughout the chapter, there were three things that were portrayed as critical to an interface’s functionality and perception. The first of these categories was font and its use to convey information through a screen. Font is super important because it conveys information to the user through it’s appearance. Fonts that are more block-like and all uppercase tend to be harder to read and mimic early style computers making the user seem more intelligent or knowledgeable. In addition, fonts that have more character like serif fonts are more readable and can be used to convey very important information that needs to be read. The size of the font is also important providing insight into the importance of the information. All of these characteristics add up to convey information about the importance and urgency of the words being displayed without the user even having to read them.

In addition to font, color is also stressed as a very important visual aspect when it comes to displays. One aspect mentioned throughout chapter 3 is the idea of the blue and glow phenomena that conveys futuristic text. This is interesting because whenever I hear blue and glow I think of hologram which is very futuristic highlighting the cultural implications of this color. In addition, colors like red on a display signify more urgent or error-like messages. While this isn’t always the case, red is described as having a sense of danger. This is an interesting interpretation because red in other cultures also signifies good luck or happiness which is generally thought of to be the opposite of danger. This difference in color perception is very interesting because it brings up the idea that it is pretty much impossible to create a perfectly perceptible user interface. While we can come to a general consensus on what means what, it is very hard to create a display that means the same thing regardless of culture especially when it comes to color schemes.

Finally, in addition to color and font size, one large aspect of displays is the user interaction through buttons or cursors. One example brought up in chapter 3 is the Star Trek LCARS interface that uses shadows to give the digital buttons on the screen a sort of depth and animation. This not only makes them feel more life-like but also gives feedback as to whether the button was pressed or not. These types of visual cues that are missed when translating from physical to digital interfaces are key in user interaction and is something that can contribute to a device being amazing or completely unusable.

Chapter 4:

Volumetric projections are very good at portraying information in such a way that humans are comfortable with. Since we see the world in 3D, VP’s allow the user to interact with the information in such a way that mimics their natural perspective. Where this can become an issue, however, is in the realness of the projection. As discussed in chapter 4, VP’s can be very tricky if they are made too realistic because the user might actually think there is something there when in reality there isn’t. While this isn’t all bad for most applications, when something that has to be physical for the safety of the user is made virtual (like a step or floor) this can create circumstances ready for injury.  In addition to safety and trickery, VP’s also have to portray the 3D information in such a way that looks natural and mimics the proportions of regular space. Otherwise the information might seem very abnormal and not as informative.

Chapter 5:

Gestural interfaces are very intuitive for the most part but also have the caveat of moving the entire body to do one single action. While there are seven distinct motions set by physical intuition but also movies and TV, these motions require sustained use of the arms at heights around heart level requiring lots of strength and endurance to keep up. This results in the user getting very tired after a small amount of use. So while gestural interfaces might seem cool, they are mostly just used for futuristic looking interfaces.

Description:

This project was meant to convey emotions through sound by playing a song corresponding to the user’s emotions when a button is pressed. Currently there are only 3 buttons for 3 different sounds/songs but the idea is to be able to hook up more and more as songs stick out as conveying your emotions. The main components for this build are buttons, resistors, a speaker and amplifier, and the Teensy 4.1. When each button is pressed, a certain song corresponding to a different emotion will start to play through. When the song is over, it waits for the next button to be pressed and does not repeat the current sound. When each button is pressed, the current song being played will stop playing and switch to the next song corresponding to the button press. As of now, button 0 plays the Jaws intense sound that signifies a stress and anxiety. Button 1 plays an African Safari game sound that signifies being happy and adventurous. Button 2 plays a relaxing song to convey peace and relaxation.

Video:

Demo Video

Images:

Code:

/*
 * Assignment 7: Sound with Meaning
 * Judson Kyle
 * judsonk
 * 
 * Description: This sketch plays a sound corresponding to the users mood at the
 *              time. Depending on the button pressed, a sound corresponding to
 *              one of three emotions will play. These emotions are intense/stressed,
 *              happy/upbeat, and peaceful/relaxed corresponding to buttons 0, 1,
 *              and 2 respectively. Each sound will only play once through to the
 *              end and not repeat after that.
 */

#include <Audio.h>
#include <Wire.h>
#include <SPI.h>
#include <SD.h>
#include <SerialFlash.h>

AudioPlaySdWav           playWav1;
AudioMixer4              mixer1;         //xy=647,408
AudioOutputMQS           mqs1;           //xy=625,279
AudioConnection          patchCord1(playWav1, 0, mqs1, 0);
AudioConnection          patchCord2(playWav1, 1, mqs1, 1);

#define SDCARD_CS_PIN    BUILTIN_SDCARD
#define SDCARD_MOSI_PIN  11  // not actually used
#define SDCARD_SCK_PIN   13  // not actually used

#define SWITCH0   33
#define SWITCH1   34
#define SWITCH2   35
#define SWITCH3   37
#define SWITCH4   38
#define SWITCH5   39

#define BUTTON0   16
#define BUTTON1   17
#define BUTTON2   18

unsigned long currTime = 0;
unsigned long debounceTime = 0;
volatile unsigned long startTime = 0;

volatile bool DEBOUNCE = false;

int numButtons = 3;

volatile int state = 0;

const char* currFileName = "Shark Attack Shortened.WAV";

void setup(void)
{
  // Wait for at least 3 seconds for the USB serial connection
  Serial.begin (9600);

  AudioMemory(8);

  pinMode(SWITCH0, INPUT);
  pinMode(SWITCH1, INPUT);
  pinMode(SWITCH2, INPUT);
  pinMode(SWITCH3, INPUT);
  pinMode(SWITCH4, INPUT);
  pinMode(SWITCH5, INPUT);

  attachInterrupt(digitalPinToInterrupt(BUTTON0), button0Pressed, CHANGE);
  attachInterrupt(digitalPinToInterrupt(BUTTON1), button1Pressed, CHANGE);
  attachInterrupt(digitalPinToInterrupt(BUTTON2), button2Pressed, CHANGE);

  SPI.setMOSI(SDCARD_MOSI_PIN);
  SPI.setSCK(SDCARD_SCK_PIN);
  if (!(SD.begin(SDCARD_CS_PIN))) {
    // stop here, but print a message repetitively
    while (1) {
      Serial.println("Unable to access the SD card");
      delay(500);
    }
  }

}

void loop (void)
{
  currTime = millis();
  static int prevState = 0;

  if (prevState != state) {
    if (playWav1.isPlaying()) {
      playWav1.stop();
    }
    switch (state) {
      case 0: //Play intense sound
        currFileName = "Shark Attack Shortened.WAV";
        playWav1.play(currFileName);
        printPlayState(currFileName);
        delay(5);
        break;
      case 1: //Play fun/upbeat sound
        currFileName = "African_fun_long.WAV";
        playWav1.play(currFileName);
        printPlayState(currFileName);
        delay(5);
        break;
      case 2: //Play peaceful sound
        currFileName = "With-You-in-My-Arms-SSJ011001.WAV";
        playWav1.play(currFileName);
        printPlayState(currFileName);
        delay(5);
        break;
    }
  }
  prevState = state;
  
  //Update debounce state
  DEBOUNCE = abs(currTime - startTime) < debounceTime;
}

//Button 0 interrupt function
void button0Pressed() {
  if ((digitalRead(BUTTON0) < 1) && !DEBOUNCE) {
    state = 0;
    startTime = currTime;
    DEBOUNCE = true;
  }
}

//Button 1 interrupt function
void button1Pressed() {
  if ((digitalRead(BUTTON1) < 1) && !DEBOUNCE) {
    state = 1;
    startTime = currTime;
    DEBOUNCE = true;
  }
}

//Button 2 interrupt function
void button2Pressed() {
  if ((digitalRead(BUTTON2) < 1) && !DEBOUNCE) {
    state = 2;
    startTime = currTime;
    DEBOUNCE = true;
  }
}

//Print out the current cong being played or return an error message if the song isn't being played
void printPlayState(const char* fileName) {
  if (!playWav1.isPlaying()) {
    Serial.print("Error playing: ");
    Serial.println(fileName);
    delay(5);
  }
  else {
    Serial.print("Playing: ");
    Serial.println(fileName);
    delay(5);
  }
}

Electrical Schematic:

Description:

This circuit turns the data that the sound sensor gets into an angle based off of a moving average of the amplitude of the data received. The data from the sensor captures the raw sound waves from the air and sends them to the arduino which has a very spiky and unsmooth nature. In addition, the data transitions for increasing to decreasing very rapidly so a traditional filter or tracking the change in the data does not work well at all. Instead, I have implemented a moving average method where the code tracks a certain number of data points, calculates the max and min of the data, and then uses the difference between these values as the new data. This is then multiplied by some multiplier to increase the variance and translated into an angle for a servo to travel to. The algorithm for calculating the angle works wonders while the servo isn’t connected, however, as depicted in the video, the noise of the servo creates an uncontrollable feedback look that results in the servo moving out of control. Regardless, the system responds to noise in the way that it should translating a noise in the environment into an angle for the servo to move to.

Video:

Code:

#include <Servo.h>

//Define Accelerometer Variables for pins
#define MICROPHONE    A0
#define SERVO_PIN     9

Servo servo;

double servoAngle = 0;

double alpha = 0.9;
double multiplier = 5;

//Array to hold data for moving average calculation
double speakerData[100]; // {0, 1, 2, 3, 4, 5, 6, 7, 8, 9}

double speakerLevel = 0;
double prevSpeakerLevel = 0;
double maxSpeakerLevel = 200;
double minSpeakerLevel = 50;


void setup() {
  //Setup pins as inputs
  pinMode(MICROPHONE, INPUT);

  //Create servo object
  servo.attach(SERVO_PIN);
  
}

void loop() {

  //Update moving average array
  for (int i = (sizeof(speakerData)/sizeof(speakerData[0]) - 1); i >= 0; i--) {
    if (i != 0) {
      speakerData[i] = speakerData[i - 1];
    }
    else {
      speakerData[i] = analogRead(MICROPHONE);
    }
  }

  double MIN = findMin();
  double MAX = findMax();

  speakerLevel = MAX - MIN;
  speakerLevel = multiplier*(alpha*speakerLevel + (1 - alpha)*prevSpeakerLevel);  //Filter data

  //Put upper and lower bounds on data
  if (speakerLevel >= maxSpeakerLevel) {
    speakerLevel = maxSpeakerLevel;
  }
  else if (speakerLevel <= minSpeakerLevel) {
    speakerLevel = minSpeakerLevel;
  }

  servoAngle = 180*(speakerLevel - minSpeakerLevel)/(maxSpeakerLevel - minSpeakerLevel);
  servo.write((int) servoAngle);

  prevSpeakerLevel = speakerLevel;
  
}

int findMin() {
  double minimum = 0;
  for (int i = 0; i < sizeof(speakerData)/sizeof(speakerData[0]); i++) {
    if (speakerData[i] < minimum || i == 0) {
      minimum = speakerData[i];
    }
  }
  return minimum;
}

int findMax() {
  double maximum = 0;
  for (int i = 0; i < sizeof(speakerData)/sizeof(speakerData[0]); i++) {
    if (speakerData[i] > maximum) {
      maximum = speakerData[i];
    }
  }
  return maximum;
}