Author: Jet Townsend

Great posts about how you react to sound.  This is what we’re looking to do with sound — bypass thinking about what you’re hearing and have an immediate reaction.

Sound crit moved to 10 Nov.   3 Nov will be a workday in A10, 5 Nov will be another workday-not-in-A10.  I’ll be on zoom for both classes.

Notes

We’re looking at sounds-out-of-context.  Experimental music, the use of new technology to invent new music, etc.

Experimental / avant-garde sound

Mark Applebaum’s experimental instruments and scoring. https://www.youtube.com/watch?v=46w99bZ3W_M

Nikoli Voinov who composed music by drawing on paper, creating animation that made sound. https://www.youtube.com/watch?v=Z7Zb4rso82M

Musique concrete using early technology to record and modify sound, including the original soundtrack to “Doctor Who”.  Some examples of recording found sound and reusing it for music.  https://www.youtube.com/watch?v=c4ea0sBrw6M

Some music you should listen to as background music while you’re doing other tasks. Avant-garde and futurism is a rather wide grouping, like saying “rock” or “country”:

Experimental composition with traditional instruments  https://www.youtube.com/watch?v=TTUXhLeLojA including John Cage: https://www.youtube.com/watch?v=pq0a317mk30

Luigi Russolo’s “industrial music” using sirens  https://www.youtube.com/watch?v=BYPXAo1cOA4

The Development of Hip-Hop

Using street technology to change and create new genres of music. Entertainment and environmental sounds can come from other contexts with the use of equipment to record, store, modify, and replay.
Turntables used to create hip hop https://www.youtube.com/watch?v=oA-OpvH4CIQ and the 1.5 hour documentary (good for a long flight)   https://www.youtube.com/watch?v=YEKRAn-ZleM

Appropriating Sounds from Other Domains

There are a couple of famous sounds in hip-hop that became parts of pop and rave music.

Watch these:

• The Orchestra Hit  https://www.youtube.com/watch?v=8A1Aj1_EF9Y
• The Amen Break  https://www.youtube.com/watch?v=5SaFTm2bcac

Save this for free time:

Rob Base teaching DJ at the university level https://www.youtube.com/watch?v=spto6rbvvV0

Why is sound important?

Why is sound important? How do you close your earlids when you go to sleep?

How do we learn to make sounds as children?  https://www.youtube.com/watch?v=8fHi36dvTdE&feature=youtu.be&t=920

Close your eyes after following links in this section. don’t worry about the visual details and information, this is learning to understand sound and signals

Classes of sounds

Signals and alerts

Signals and alerts are short sounds that transfer a small amount of information

• automobile warning buzzers and interior sounds that communicate the state of the vehicle: https://www.soundsnap.com/tags/car_interiors
• walk/don’t walk beep boop and how crazy we were about it at CMU:  https://www.youtube.com/watch?v=dcE6gn-JLbw
• fire department / police codes on dispatch channels:  http://www.policeinterceptor.com/emerg.htm
• collection that includes a game tone https://www.zedge.net/find/ringtones/dispatch
• doorbell: https://www.youtube.com/watch?v=sbCyxgLgmEE https://www.youtube.com/watch?v=fUVj0BnC8Gw
• ringing phone: https://www.youtube.com/watch?v=3GnoCz5mw3M

Information over time – songs and patterns, more complex information, many of them skeuomorphic

• air raid siren, dual pitch: https://www.youtube.com/watch?v=cURcd2_w-rg
• tornado sirens: https://www.youtube.com/watch?v=Djqi86jMl5g
  • my borough uses this as the 15 minute warning on curfew for minors(!)

Music and entertainment

Ringing phones are so unique that we map and learn our own ringtones using songs. Golan Levin’s mobile phone concert was only possible because phones had ring tones that couldn’t be changed.  http://www.flong.com/projects/telesymphony/

Star Trek had one of the earliest catalogs of special effects sounds used to alert viewers of plot elements and activity.  https://www.mediacollege.com/downloads/sound-effects/star-trek/tos/

Professional companies that sell sound libraries. https://www.soundsnap.com/

Chili’s used the sound of sizzling fajitas to sell them, not their flavor:  https://99percentinvisible.org/episode/the-sizzle/

We like things that reflect heartbeat rhythm, this goes back to dance for thousands of years, continue to do this in today’s music with multiple tempo compositions.  Orbital and Underworld are two electronic bands using dual internal tempos, one near the rate of resting heart beat the other near the rate of active (dancing) heart beat.  https://www.youtube.com/watch?v=IcQXy4YdFcM

Psychological effects of sound

Is it genetics that cause us to respond to the sound of a crying human baby? Can you think of an “angry” noise? A “happy” noise? a “relaxing” noise?

 

I’ve added a tool that shows up in the Post editor as the icon “{…}”.   To post the code below, I clicked on that icon, pasted in my code, turned off indentation, then selected C++, and hit “OK”.

 

// -*-c++-*-
/*
  The MIT License (MIT)

  Copyright (c) 2017 J. Eric Townsend

  Permission is hereby granted, free of charge, to any person obtaining a copy
  of this software and associated documentation files (the "Software"), to deal
  in the Software without restriction, including without limitation the rights
  to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
  copies of the Software, and to permit persons to whom the Software is
  furnished to do so, subject to the following conditions:

  The above copyright notice and this permission notice shall be included in
  all copies or substantial portions of the Software.

  THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
  IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
  FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
  AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
  LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
  OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
  THE SOFTWARE.
*/

/*
  using interrupts to stop/alter a process that's eating up the loop()
cycle with delays

 */

static const int potOnPin = A1;
static const int potOffPin = A1;


static const int togglePin = 2;

static const int LED1Pin = 5;
static const int LED2Pin = 6;
static const int LED3Pin = 9;

int LEDOnDuration = 1000;  // milliseconds
int LEDOffDuration = 1000;  // milliseconds
long LEDOnTime = 0;
long LEDOffTime = 0;;
bool LEDState = false;
bool LEDStatePrev = false;
int toggleState = 0;

bool flagState = false;
const bool isInterrupt = true;

// TODO: how do we stop the for() loops so we can do another analogRead()?
void SwitchPressed()
{
    // we can set the delay to 0 and fall out of the loops
  //LEDOnDuration = 0;
  //LEDOffDuration = 0;
    // or we can modify a flag variable and use that to exit the loop
       flagState = true;
}

void setup() {
  Serial.begin(115200);

  pinMode(potOnPin, INPUT);
  pinMode(potOffPin, INPUT);
  pinMode(togglePin, INPUT);

  pinMode(LED1Pin, OUTPUT);
  pinMode(LED2Pin, OUTPUT);
  pinMode(LED3Pin, OUTPUT);

  if (isInterrupt) {
    attachInterrupt (digitalPinToInterrupt(togglePin), SwitchPressed, CHANGE);
  }
}

void loop() {

  if (!isInterrupt) {
    flagState = digitalRead(togglePin);
  }

  if (flagState) {
    Serial.println("flagState true");
  }

  // take input from the pots to determine on/off times
  int pot = analogRead(potOnPin);
  if (pot != LEDOnDuration) {
   //Serial.print("on: ");
    //Serial.println(pot);
    LEDOnDuration = map(pot, 0, 1024, 0, 255);
  }
  pot = analogRead(potOffPin);
  if (pot != LEDOffDuration) {
   // Serial.print("off: ");
   // Serial.println(pot);
    LEDOffDuration = map(pot, 0, 1024, 0, 255);
  }


  // now let's write some really bad code
  // turn the LEDs on
  for (int i = 0; i < 255; i++) {
      if (flagState) {
        break;
  // is more reliable than setting i = 256;
          // if we set flagState back to false here we
          // won't make a change in the next loop where we
          // turn LEDs off
      }
      else {
          analogWrite(LED1Pin, i);
          analogWrite(LED2Pin, i);
          analogWrite(LED3Pin, i);
          delay(LEDOnDuration);
      }
  }
  // turn the LEDs off
  for (int i = 255; i > 0; i--) {
      if (flagState) {
          break;
  // is more reliable than setting i = 256;
          // if we set flagState back to false here we
          // won't make a change in the next loop where we
          // turn LEDs on
      }
      else {
          analogWrite(LED1Pin, i);
          analogWrite(LED2Pin, i);
          analogWrite(LED3Pin, i);
          delay(LEDOffDuration);
      }
  }

  // this is the proper place to turn off the flagState.
  // We could do this with just a statement and never check
  // its status, but using a test makes debugging easier
  if (flagState == true) {
      flagState = false;
      // because we could add this
      Serial.println("flagState was on, now turned off");
  }

}

 

Kinetic input devices

In 1968, Doug Englebart demonstrated the first “workstation”.  It’s a long watch but I think at least the first half will give you a lot of ideas on how to pitch a novel technological concept.  Stanford has a broken down version in flash.

Doug Engelbart’s 1968 Demo in full from Frode Hegland on Vimeo.

Accessibility vs Inclusion

What makes something accessible?  Is universal design also accessibility?

Inclusion ==> inviting, making someone want to participate. How do you invite someone to provide input / direction?

Microsoft has a great program on

Microsoft Inclusion and PDF book

Are 30mm arcade buttons are accessible? Interrupt or constant? Convex or concave? If you want to use Universal Design, how do you decide how big the button should be and where it’s located?

There’s a wide variety of arcade push buttons.  Are controls like buttons the wrong answer?  Is a better way to collect input?

What is wrong with the E-Stop button in A10?

• Unlit
• Recessed button “hidden” in a guard
• No signage on the wall like we have with fire extinguishers

Kinetic Output and representation of data

tactile maps: https://www.youtube.com/watch?v=psObymNkzvk

research data on tactile map comparison: https://www.youtube.com/watch?v=Qbe9G9dWcSk

tactile graphics using “swell paper”: https://www.youtube.com/watch?v=QeulfaWn_Ps

what does a snowflake look like?  A butterfly?  A sailboat?

3d printing for the blind: https://www.youtube.com/watch?v=bEVTu1xdpVI

Not discussed in class — make your own 3d prints of maps.  https://www.instructables.com/Make-3d-Printed-Topo-Maps-of-Anywhere/

Interrupts and kinetic interaction

• Interrupts as inputs to state machines
• Human interruption as part of a UI
• Environmental interruption as part of communication.  How do we interrupt each other (politely!) during conference calls or video sessions?
• Interrupts that limit motion — end-stop switches

ASL interpreters and physical communication

In lecture I was talking about ASL translators for public speakers, but I think interpretations of music  and lyrics are even more dramatic

ASL Poetry Sign-Off Finals

Cardi B, Bodak Yellow in ASL

Hamilton, “Wait for it“, “Guns and Ships“, and “Alexander Hamilton” by a group with nice costumes.

 

 

Haptic/touch vs. objects moving in space

touching a person vs. moving an object

person presses a button vs. wind blowing a windmill

touching a person vs. moving an object – touch is more personal, requires little energy.

Touch can be wrapped in a robotic device, ex: Paro (wiki) trade show demo.

Moving things typically requires an external power source, Arduino can only provide 5V at a few milliamps.

energy required to move large things vs. small things

explain watts vs. amps vs voltage vs. movement

For moving things, use basic physics

• gears — FAQ on 3d printing gears
• levers — drive a lever with a solenoid, transfer short movement to long
• pulleys – drive with DC motors or stepper

Input classification, serial communication, interrupts.

Types of Input

monophonic + skill: wind instruments, percussion

polyphonic + tech: keyboards, pianos, organs, strings
anthropomorphic: respond to human condition, blood pressure, galvanic skin, breath rate, pulse rate, visual interpretation of secondary movements: eye twitch, touching your face, blinking

“rapid eye movement” when eyes are closed, used in PTSD therapy called “EMDR”:

https://en.wikipedia.org/wiki/Eye_movement_desensitization_and_reprocessing

SparkFun biometric sensors: https://www.sparkfun.com/categories/146

UPMC lobby temperature check:  upmc-lobby-temperature-check

Serial Communication

SPI/I2C and complex communications protocols
How we get complex data from sensors – a lot of this is hidden in libraries
Unique IDs
Simple controls for complex output: neopixel
SparkFun’s version: Qwiic

Interrupts

Show examples of interrupt code in the environment
switches on mobiles
remote controls for the projectors, TVs, stereos
little to no latency
complex interrupt systems in video game controllers
rotary encoder (we’ll do a demo later in the semester)
for now, we only use digital inputs for interrupts

Code samples, show how an interrupt can be used to toggle a state by one increment compared to holding down a switch and falling through a number of states.
Note that holding down the switch means the interrupt service routine (ISR) only functions once
Compare to using delay() to sample data every so many units of time.

Use an interrupt to stop a task that takes a long time, say a long for() or while() loop, by adjusting the terminating conditions

Question: What if you were playing mp3 files or video, how would you use interrupts as part of the interface?

zip file with some examples:  interrupts-1-pot

administrivia

voting – go vote!  If you think you’ll be late for class, please drop me email in advance.

celebrate non-US holidays:  Chuseok and other Fall-holidays are a perfectly fine reason to miss class

Update:  another banned assignment is navigation belts as they have been done to death.

Physical interaction with temperature

Adam Savage’s DIY costume cooling vest for cosplay and a commercial alternative.

Consider medical/physically safety with devices that touch the skin.  It’s really easy to get burned/frozen with Pletier plates and ice.  Instead of smoke, use soap bubbles and a fan.

Coaching vs. grading

Think  about coaching, providing good feedback and encouragement to take a positive action instead of negative feedback.   In Total Control Training we use positive feedback and coaching.  Instead of saying, “You did that wrong, you were too slow going in to the corner,do it again” we say “That’s good, now do it again with a little more throttle as you go in to the corner.”

You want your interaction to be one that invites people and makes them want to interact.  If you do a good job, people will wait in line to use your interaction:  https://vimeo.com/3796361

Example: sports trainer that monitors your HR, BP, breathing rate, and hydration and knows your training course.  It encourages you to do better instead of punishing you for not doing enough.

Example: music “coach” that helps you learn to perform music. Watches your body and helps you correct form/posture.  Reminds you that you are always performing, even when you’re just practicing a scale or an etude.

Something to read over the break:  Alice Miller’s “For Your Own Good“, a criticism arguing that we replace the pedagogy of punishment  with support for learning, using the German pedagogy that gave rise to support of fascism as one study.