Category: Class Notes

Here’s the p5/arduino example

p5-example.tar

Interactive jacket project from Baskinger’s Experimental Form class.

Look at the 2019 Final Crit for some good examples on elevator pitches and proposals.

Next Tuesday, 24 Nov, is the last day you can use A10.  We’ll make that a work day.  Thursday, 26 Nov, no class due to turkey day.

For Thursday, have a proposal posted to the blog’s Final Crit category.

 

p5.js / arduino

Configure/test p5.js not required for final crit but lets make sure you can use it if you decide to.  The NYU ITP instructions are a good guide.  Do not use the online p5.js web editor, use local files and a local browser and text editor.

Final Crit

Start thinking about the final crit. We only have one more day to access A10, 24 Nov, and you should have your wish lists ready

For the final crit  combine visual, sound, and kinetics with a state machine and sensors to make something more accessible, usable, or uses universal design.

Here’s a rough schedule.  On some of the work days I will do a brief lecture based on questions or inspirational projects done in other physical computing projects.  The remainder of this time is making up for my not having open-office hours in A10.

• 17 Nov — workshop proposals
• 19 Nov — work day
• 24 Nov — last day in A10
• 26 Nov — vacation
• 1 Dec — work day
• 3 Dec — crit test flight
• 8 Dec — work day
• 10 Dec — final crit

Assignment for Tuesday

Crit warmup: come up with a problem to solve.  No need to write code (unless you want to), just a statement of the problem and your solution.

We will discuss these on Tuesday and brainstorm improvements.

Viewing assignment

These are based on the idea of how you can hide/obfuscate the man behind the curtain.

Former FBI Agent Explains How to Read Body Language

Former CIA Operative In Charge of Disguise Explains How Spies Use Disguises  (except for the model with a sleeve)

…and debunks spies and disguises in entertainment:

Chapters 11 and 12

Pretty simple circuit that makes a lot of noise.

PDF of the sketch arduino-transistor-speaker

Genre conflation

This is a counter to sampling and using electronics to invent new music. Instruments from one style of music are used to perform a style of music from a completely unrelated genre. My favorite examples are “pirate metal” or the band Orkestra Obsolete playing famous pop music.

We’re skipping a lot of contemporary genres — things like EDM, IDM, dub, liquid trap, and future soul. There’s an entire class in this topic, but if you want to go on a genre-binge-fest then Ishkur’s Guide is one place to start: http://music.ishkur.com/

Physical interaction options

How to hide things in plain sight: https://www.youtube.com/watch?v=T61-twuvJA4

Bill Shannon, PGH native, who defies disability with dance performance:

 

and his tutorial on a dance form he developed:

Ways of making sound

Some examples of different ways people create sound

Using things you already have at home

There’s a Japanese TV show for kids, “Pythagora Switch“, that features Rube-Goldberg like devices made with stuff you’d find around the house. These are so popular with adults that people go a little over the top making/recording their own devices still using common items.

Arduino data collection/processing tips

how to debounce a switch: https://www.arduino.cc/en/Tutorial/Debounce

standard deviations, arrays, and monitoring a variable source. flattening data with mean / median.

// -*-c++-*-
/*
  The MIT License (MIT)

  Copyright (c) 2019 Jet Townsend

  Permission is hereby granted, free of charge, to any person obtaining a copy
  of this software and associated documentation files (the "Software"), to deal
  in the Software without restriction, including without limitation the rights
  to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
  copies of the Software, and to permit persons to whom the Software is
  furnished to do so, subject to the following conditions:

  The above copyright notice and this permission notice shall be included in
  all copies or substantial portions of the Software.

  THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
  IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
  FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
  AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
  LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
  OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
  THE SOFTWARE.
*/

/* basic stats with analogRead()

   intputs
   A0 pot

   output
   D2 Speaker
   D13 LED

   40 NEOPIXEL

*/

#include <Adafruit_NeoPixel.h>

#ifdef __AVR__
#include <avr/power.h>
#endif
#define PIN            44
#define NUMPIXELS      1
Adafruit_NeoPixel pixels = Adafruit_NeoPixel(NUMPIXELS, PIN, NEO_GRBW + NEO_KHZ800);

static int plotter = 1;
static int console = !plotter;

static const int inPot = A0;
unsigned int potValue;

static const int sampleSize = 16;
int readings[sampleSize];
unsigned int sampleIndex = 0;
unsigned long lastSampleTime = 0;
unsigned long sampleInterval = 100; // in ms
// we need to move these out of the loop() as well
unsigned int median = 0;
unsigned int low = 1024;
unsigned int high = 0;
unsigned int mean = 0;

static const int statusLed = 13;
unsigned int lastRedVal = 0;
unsigned int analogMax = 1023;

void setup() {
  //analogWriteResolution(10); // Set analog out resolution to max, 10-bits
  analogReadResolution(12); // and read at 12 bits
  analogMax = 1023 * 4;

  Serial.begin(115200);

  pinMode(inPot, INPUT);

  pinMode(statusLed, OUTPUT);

  pixels.begin(); // This initializes the NeoPixel library.

}

void loop() {
  // TODO what happens if you try to set a negative color?
  unsigned int redVal = 0;

  // TODO
  // don't read the input on every loop to help stabilize the data
  // sampling over time is also a UI change, you have to actively
  // block light for
  // sampleSize * sampleInterval milliseconds to create a change

  unsigned long now = millis();
  if (lastSampleTime + sampleInterval < now) {
    lastSampleTime = now;
     potValue = analogRead(inPot);

    // good for debugging:
    //        Serial.print(now);
    //        Serial.print(" ");
    //        Serial.println(potValue);

    readings[sampleIndex] = potValue;
    sampleIndex < (sampleSize - 1) ? sampleIndex++ : sampleIndex = 0;

    digitalWrite(inPot, HIGH);
    for (int i = 0; i < sampleSize; i++) {
      mean += readings[i];
      if (readings[i] < low) {
        low = readings[i];
      }
      if (readings[i] > high) {
        high = readings[i];
      }
    }
    mean = mean / sampleSize;
    bubbleSort();
    median = readings[sampleSize / 2];
  }

  // set the pixels with different values, mean, median, current read

  // long map(long x, long in_min, long in_max, long out_min, long out_max)
  // in A10, the value of 600 was the bottom end
  // TODO use a flashlight to break this, more light == lower number
  // than 600
  redVal = map(median, 5, analogMax, 0, 255);

  //  at home, 950 was the bottom end. Remember that higher value == less light
  // how do we dynamically set the bottom end of the map?
  //    redVal = map(median, 950, analogMax, 0, 255);

  // what about changing the max value of redVal so we have a dimmer range?
  //    redVal = map(median, 600, analogMax, 0, 128);

  // TODO keep the pixel from blinking
  /*
    // simple, but allows for some blinking
      if (redVal != lastRedVal) {
          pixels.setPixelColor(0, pixels.Color(redVal,0,0));
          pixels.show(); // This sends the updated pixel color to the hardware.
          lastRedVal = redVal;
      }
  */


  // use a tolerance to limit to a range
  // by making this not a const we can change it in the code
  // as needed

  int tolerance = 5;

  // This looks like it should work, right?
  // what happens if lastRedVal is 0?
  if (
    (redVal >  (lastRedVal + tolerance))
    || (redVal <  (lastRedVal - tolerance))
  ) {

    pixels.setPixelColor(0, pixels.Color(redVal, 0, 0));
    pixels.show(); // This sends the updated pixel color to the hardware.
    lastRedVal = redVal;
  }

  // console
  if (console) {
    Serial.print("reading ");
    Serial.print(potValue);
    Serial.print(" low ");
    Serial.print(low);
    Serial.print(" high ");
    Serial.print(high);
    Serial.print(" mean ");
    Serial.println(mean);
    Serial.print(" median ");
    Serial.print(median);
    Serial.print(",");
    Serial.print(" redVal ");
    Serial.println(redVal);
    delay(100);
    digitalWrite(inPot, LOW);
    delay(100);
  }
  else if (plotter) {
    // plotter
    Serial.print(low);
    Serial.print(",");
    Serial.print(high);
    Serial.print(",");
    Serial.print(mean);
    Serial.print(",");
    Serial.print(median);
    Serial.print(",");
    Serial.println(redVal);
  }

}

// tigoe's sort for an array
void bubbleSort() {
  int out, in, swapper;
  for (out = 0 ; out < sampleSize; out++) { // outer loop
    for (in = out; in < (sampleSize - 1); in++)  { // inner loop
      if ( readings[in] > readings[in + 1] ) { // out of order?
        // swap them:
        swapper = readings[in];
        readings [in] = readings[in + 1];
        readings[in + 1] = swapper;
      }
    }
  }
}

 

 

Great posts about how you react to sound.  This is what we’re looking to do with sound — bypass thinking about what you’re hearing and have an immediate reaction.

Sound crit moved to 10 Nov.   3 Nov will be a workday in A10, 5 Nov will be another workday-not-in-A10.  I’ll be on zoom for both classes.

Notes

We’re looking at sounds-out-of-context.  Experimental music, the use of new technology to invent new music, etc.

Experimental / avant-garde sound

Mark Applebaum’s experimental instruments and scoring. https://www.youtube.com/watch?v=46w99bZ3W_M

Nikoli Voinov who composed music by drawing on paper, creating animation that made sound. https://www.youtube.com/watch?v=Z7Zb4rso82M

Musique concrete using early technology to record and modify sound, including the original soundtrack to “Doctor Who”.  Some examples of recording found sound and reusing it for music.  https://www.youtube.com/watch?v=c4ea0sBrw6M

Some music you should listen to as background music while you’re doing other tasks. Avant-garde and futurism is a rather wide grouping, like saying “rock” or “country”:

Experimental composition with traditional instruments  https://www.youtube.com/watch?v=TTUXhLeLojA including John Cage: https://www.youtube.com/watch?v=pq0a317mk30

Luigi Russolo’s “industrial music” using sirens  https://www.youtube.com/watch?v=BYPXAo1cOA4

The Development of Hip-Hop

Using street technology to change and create new genres of music. Entertainment and environmental sounds can come from other contexts with the use of equipment to record, store, modify, and replay.
Turntables used to create hip hop https://www.youtube.com/watch?v=oA-OpvH4CIQ and the 1.5 hour documentary (good for a long flight)   https://www.youtube.com/watch?v=YEKRAn-ZleM

Appropriating Sounds from Other Domains

There are a couple of famous sounds in hip-hop that became parts of pop and rave music.

Watch these:

• The Orchestra Hit  https://www.youtube.com/watch?v=8A1Aj1_EF9Y
• The Amen Break  https://www.youtube.com/watch?v=5SaFTm2bcac

Save this for free time:

Rob Base teaching DJ at the university level https://www.youtube.com/watch?v=spto6rbvvV0

Why is sound important?

Why is sound important? How do you close your earlids when you go to sleep?

How do we learn to make sounds as children?  https://www.youtube.com/watch?v=8fHi36dvTdE&feature=youtu.be&t=920

Close your eyes after following links in this section. don’t worry about the visual details and information, this is learning to understand sound and signals

Classes of sounds

Signals and alerts

Signals and alerts are short sounds that transfer a small amount of information

• automobile warning buzzers and interior sounds that communicate the state of the vehicle: https://www.soundsnap.com/tags/car_interiors
• walk/don’t walk beep boop and how crazy we were about it at CMU:  https://www.youtube.com/watch?v=dcE6gn-JLbw
• fire department / police codes on dispatch channels:  http://www.policeinterceptor.com/emerg.htm
• collection that includes a game tone https://www.zedge.net/find/ringtones/dispatch
• doorbell: https://www.youtube.com/watch?v=sbCyxgLgmEE https://www.youtube.com/watch?v=fUVj0BnC8Gw
• ringing phone: https://www.youtube.com/watch?v=3GnoCz5mw3M

Information over time – songs and patterns, more complex information, many of them skeuomorphic

• air raid siren, dual pitch: https://www.youtube.com/watch?v=cURcd2_w-rg
• tornado sirens: https://www.youtube.com/watch?v=Djqi86jMl5g
  • my borough uses this as the 15 minute warning on curfew for minors(!)

Music and entertainment

Ringing phones are so unique that we map and learn our own ringtones using songs. Golan Levin’s mobile phone concert was only possible because phones had ring tones that couldn’t be changed.  http://www.flong.com/projects/telesymphony/

Star Trek had one of the earliest catalogs of special effects sounds used to alert viewers of plot elements and activity.  https://www.mediacollege.com/downloads/sound-effects/star-trek/tos/

Professional companies that sell sound libraries. https://www.soundsnap.com/

Chili’s used the sound of sizzling fajitas to sell them, not their flavor:  https://99percentinvisible.org/episode/the-sizzle/

We like things that reflect heartbeat rhythm, this goes back to dance for thousands of years, continue to do this in today’s music with multiple tempo compositions.  Orbital and Underworld are two electronic bands using dual internal tempos, one near the rate of resting heart beat the other near the rate of active (dancing) heart beat.  https://www.youtube.com/watch?v=IcQXy4YdFcM

Psychological effects of sound

Is it genetics that cause us to respond to the sound of a crying human baby? Can you think of an “angry” noise? A “happy” noise? a “relaxing” noise?

 

I’ve added a tool that shows up in the Post editor as the icon “{…}”.   To post the code below, I clicked on that icon, pasted in my code, turned off indentation, then selected C++, and hit “OK”.

 

// -*-c++-*-
/*
  The MIT License (MIT)

  Copyright (c) 2017 J. Eric Townsend

  Permission is hereby granted, free of charge, to any person obtaining a copy
  of this software and associated documentation files (the "Software"), to deal
  in the Software without restriction, including without limitation the rights
  to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
  copies of the Software, and to permit persons to whom the Software is
  furnished to do so, subject to the following conditions:

  The above copyright notice and this permission notice shall be included in
  all copies or substantial portions of the Software.

  THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
  IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
  FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
  AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
  LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
  OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
  THE SOFTWARE.
*/

/*
  using interrupts to stop/alter a process that's eating up the loop()
cycle with delays

 */

static const int potOnPin = A1;
static const int potOffPin = A1;


static const int togglePin = 2;

static const int LED1Pin = 5;
static const int LED2Pin = 6;
static const int LED3Pin = 9;

int LEDOnDuration = 1000;  // milliseconds
int LEDOffDuration = 1000;  // milliseconds
long LEDOnTime = 0;
long LEDOffTime = 0;;
bool LEDState = false;
bool LEDStatePrev = false;
int toggleState = 0;

bool flagState = false;
const bool isInterrupt = true;

// TODO: how do we stop the for() loops so we can do another analogRead()?
void SwitchPressed()
{
    // we can set the delay to 0 and fall out of the loops
  //LEDOnDuration = 0;
  //LEDOffDuration = 0;
    // or we can modify a flag variable and use that to exit the loop
       flagState = true;
}

void setup() {
  Serial.begin(115200);

  pinMode(potOnPin, INPUT);
  pinMode(potOffPin, INPUT);
  pinMode(togglePin, INPUT);

  pinMode(LED1Pin, OUTPUT);
  pinMode(LED2Pin, OUTPUT);
  pinMode(LED3Pin, OUTPUT);

  if (isInterrupt) {
    attachInterrupt (digitalPinToInterrupt(togglePin), SwitchPressed, CHANGE);
  }
}

void loop() {

  if (!isInterrupt) {
    flagState = digitalRead(togglePin);
  }

  if (flagState) {
    Serial.println("flagState true");
  }

  // take input from the pots to determine on/off times
  int pot = analogRead(potOnPin);
  if (pot != LEDOnDuration) {
   //Serial.print("on: ");
    //Serial.println(pot);
    LEDOnDuration = map(pot, 0, 1024, 0, 255);
  }
  pot = analogRead(potOffPin);
  if (pot != LEDOffDuration) {
   // Serial.print("off: ");
   // Serial.println(pot);
    LEDOffDuration = map(pot, 0, 1024, 0, 255);
  }


  // now let's write some really bad code
  // turn the LEDs on
  for (int i = 0; i < 255; i++) {
      if (flagState) {
        break;
  // is more reliable than setting i = 256;
          // if we set flagState back to false here we
          // won't make a change in the next loop where we
          // turn LEDs off
      }
      else {
          analogWrite(LED1Pin, i);
          analogWrite(LED2Pin, i);
          analogWrite(LED3Pin, i);
          delay(LEDOnDuration);
      }
  }
  // turn the LEDs off
  for (int i = 255; i > 0; i--) {
      if (flagState) {
          break;
  // is more reliable than setting i = 256;
          // if we set flagState back to false here we
          // won't make a change in the next loop where we
          // turn LEDs on
      }
      else {
          analogWrite(LED1Pin, i);
          analogWrite(LED2Pin, i);
          analogWrite(LED3Pin, i);
          delay(LEDOffDuration);
      }
  }

  // this is the proper place to turn off the flagState.
  // We could do this with just a statement and never check
  // its status, but using a test makes debugging easier
  if (flagState == true) {
      flagState = false;
      // because we could add this
      Serial.println("flagState was on, now turned off");
  }

}

 

Kinetic input devices

In 1968, Doug Englebart demonstrated the first “workstation”.  It’s a long watch but I think at least the first half will give you a lot of ideas on how to pitch a novel technological concept.  Stanford has a broken down version in flash.

Doug Engelbart’s 1968 Demo in full from Frode Hegland on Vimeo.

Accessibility vs Inclusion

What makes something accessible?  Is universal design also accessibility?

Inclusion ==> inviting, making someone want to participate. How do you invite someone to provide input / direction?

Microsoft has a great program on

Microsoft Inclusion and PDF book

Are 30mm arcade buttons are accessible? Interrupt or constant? Convex or concave? If you want to use Universal Design, how do you decide how big the button should be and where it’s located?

There’s a wide variety of arcade push buttons.  Are controls like buttons the wrong answer?  Is a better way to collect input?

What is wrong with the E-Stop button in A10?

• Unlit
• Recessed button “hidden” in a guard
• No signage on the wall like we have with fire extinguishers

Kinetic Output and representation of data

tactile maps: https://www.youtube.com/watch?v=psObymNkzvk

research data on tactile map comparison: https://www.youtube.com/watch?v=Qbe9G9dWcSk

tactile graphics using “swell paper”: https://www.youtube.com/watch?v=QeulfaWn_Ps

what does a snowflake look like?  A butterfly?  A sailboat?

3d printing for the blind: https://www.youtube.com/watch?v=bEVTu1xdpVI

Not discussed in class — make your own 3d prints of maps.  https://www.instructables.com/Make-3d-Printed-Topo-Maps-of-Anywhere/

Interrupts and kinetic interaction

• Interrupts as inputs to state machines
• Human interruption as part of a UI
• Environmental interruption as part of communication.  How do we interrupt each other (politely!) during conference calls or video sessions?
• Interrupts that limit motion — end-stop switches

ASL interpreters and physical communication

In lecture I was talking about ASL translators for public speakers, but I think interpretations of music  and lyrics are even more dramatic

ASL Poetry Sign-Off Finals

Cardi B, Bodak Yellow in ASL

Hamilton, “Wait for it“, “Guns and Ships“, and “Alexander Hamilton” by a group with nice costumes.