Here’s the p5/arduino example
Category: Class Notes
Class notes, 17 Nov 2020
Interactive jacket project from Baskinger’s Experimental Form class.
Look at the 2019 Final Crit for some good examples on elevator pitches and proposals.
Next Tuesday, 24 Nov, is the last day you can use A10. We’ll make that a work day. Thursday, 26 Nov, no class due to turkey day.
For Thursday, have a proposal posted to the blog’s Final Crit category.
Class notes, 12 Nov 2020
p5.js / arduino
Configure/test p5.js not required for final crit but lets make sure you can use it if you decide to. The NYU ITP instructions are a good guide. Do not use the online p5.js web editor, use local files and a local browser and text editor.
Final Crit
Start thinking about the final crit. We only have one more day to access A10, 24 Nov, and you should have your wish lists ready
For the final crit combine visual, sound, and kinetics with a state machine and sensors to make something more accessible, usable, or uses universal design.
Here’s a rough schedule. On some of the work days I will do a brief lecture based on questions or inspirational projects done in other physical computing projects. The remainder of this time is making up for my not having open-office hours in A10.
- 17 Nov — workshop proposals
- 19 Nov — work day
- 24 Nov — last day in A10
- 26 Nov — vacation
- 1 Dec — work day
- 3 Dec — crit test flight
- 8 Dec — work day
- 10 Dec — final crit
Assignment for Tuesday
Crit warmup: come up with a problem to solve. No need to write code (unless you want to), just a statement of the problem and your solution.
We will discuss these on Tuesday and brainstorm improvements.
Viewing assignment
These are based on the idea of how you can hide/obfuscate the man behind the curtain.
Former FBI Agent Explains How to Read Body Language
Former CIA Operative In Charge of Disguise Explains How Spies Use Disguises (except for the model with a sleeve)
…and debunks spies and disguises in entertainment:
quick reading assignment for 12 Nov
Chapters 11 and 12
Using a transducer to make my desktop a speaker
Pretty simple circuit that makes a lot of noise.
PDF of the sketch arduino-transistor-speaker
Class notes, 27 Sep 2020
Genre conflation
This is a counter to sampling and using electronics to invent new music. Instruments from one style of music are used to perform a style of music from a completely unrelated genre. My favorite examples are “pirate metal” or the band Orkestra Obsolete playing famous pop music.
We’re skipping a lot of contemporary genres — things like EDM, IDM, dub, liquid trap, and future soul. There’s an entire class in this topic, but if you want to go on a genre-binge-fest then Ishkur’s Guide is one place to start: http://music.ishkur.com/
Physical interaction options
How to hide things in plain sight: https://www.youtube.com/watch?v=T61-twuvJA4
Bill Shannon, PGH native, who defies disability with dance performance:
and his tutorial on a dance form he developed:
Ways of making sound
Some examples of different ways people create sound
Foley artists creating the sounds of weather.
The Arduino guide to handling debounce on switches.
A mapping of western scale notes to frequencies.
The Sound Noise Device that uses cameras to create animation and sound.
It’s easy to create sounds without speakers:
- music made with steppers
- A “top ten” list of arduino music projects
- historical tube doorbells and someone what lost their mind
Using things you already have at home
There’s a Japanese TV show for kids, “Pythagora Switch“, that features Rube-Goldberg like devices made with stuff you’d find around the house. These are so popular with adults that people go a little over the top making/recording their own devices still using common items.
Arduino data collection/processing tips
how to debounce a switch: https://www.arduino.cc/en/Tutorial/Debounce
standard deviations, arrays, and monitoring a variable source. flattening data with mean / median.
// -*-c++-*- /* The MIT License (MIT) Copyright (c) 2019 Jet Townsend Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. */ /* basic stats with analogRead() intputs A0 pot output D2 Speaker D13 LED 40 NEOPIXEL */ #include <Adafruit_NeoPixel.h> #ifdef __AVR__ #include <avr/power.h> #endif #define PIN 44 #define NUMPIXELS 1 Adafruit_NeoPixel pixels = Adafruit_NeoPixel(NUMPIXELS, PIN, NEO_GRBW + NEO_KHZ800); static int plotter = 1; static int console = !plotter; static const int inPot = A0; unsigned int potValue; static const int sampleSize = 16; int readings[sampleSize]; unsigned int sampleIndex = 0; unsigned long lastSampleTime = 0; unsigned long sampleInterval = 100; // in ms // we need to move these out of the loop() as well unsigned int median = 0; unsigned int low = 1024; unsigned int high = 0; unsigned int mean = 0; static const int statusLed = 13; unsigned int lastRedVal = 0; unsigned int analogMax = 1023; void setup() { //analogWriteResolution(10); // Set analog out resolution to max, 10-bits analogReadResolution(12); // and read at 12 bits analogMax = 1023 * 4; Serial.begin(115200); pinMode(inPot, INPUT); pinMode(statusLed, OUTPUT); pixels.begin(); // This initializes the NeoPixel library. } void loop() { // TODO what happens if you try to set a negative color? unsigned int redVal = 0; // TODO // don't read the input on every loop to help stabilize the data // sampling over time is also a UI change, you have to actively // block light for // sampleSize * sampleInterval milliseconds to create a change unsigned long now = millis(); if (lastSampleTime + sampleInterval < now) { lastSampleTime = now; potValue = analogRead(inPot); // good for debugging: // Serial.print(now); // Serial.print(" "); // Serial.println(potValue); readings[sampleIndex] = potValue; sampleIndex < (sampleSize - 1) ? sampleIndex++ : sampleIndex = 0; digitalWrite(inPot, HIGH); for (int i = 0; i < sampleSize; i++) { mean += readings[i]; if (readings[i] < low) { low = readings[i]; } if (readings[i] > high) { high = readings[i]; } } mean = mean / sampleSize; bubbleSort(); median = readings[sampleSize / 2]; } // set the pixels with different values, mean, median, current read // long map(long x, long in_min, long in_max, long out_min, long out_max) // in A10, the value of 600 was the bottom end // TODO use a flashlight to break this, more light == lower number // than 600 redVal = map(median, 5, analogMax, 0, 255); // at home, 950 was the bottom end. Remember that higher value == less light // how do we dynamically set the bottom end of the map? // redVal = map(median, 950, analogMax, 0, 255); // what about changing the max value of redVal so we have a dimmer range? // redVal = map(median, 600, analogMax, 0, 128); // TODO keep the pixel from blinking /* // simple, but allows for some blinking if (redVal != lastRedVal) { pixels.setPixelColor(0, pixels.Color(redVal,0,0)); pixels.show(); // This sends the updated pixel color to the hardware. lastRedVal = redVal; } */ // use a tolerance to limit to a range // by making this not a const we can change it in the code // as needed int tolerance = 5; // This looks like it should work, right? // what happens if lastRedVal is 0? if ( (redVal > (lastRedVal + tolerance)) || (redVal < (lastRedVal - tolerance)) ) { pixels.setPixelColor(0, pixels.Color(redVal, 0, 0)); pixels.show(); // This sends the updated pixel color to the hardware. lastRedVal = redVal; } // console if (console) { Serial.print("reading "); Serial.print(potValue); Serial.print(" low "); Serial.print(low); Serial.print(" high "); Serial.print(high); Serial.print(" mean "); Serial.println(mean); Serial.print(" median "); Serial.print(median); Serial.print(","); Serial.print(" redVal "); Serial.println(redVal); delay(100); digitalWrite(inPot, LOW); delay(100); } else if (plotter) { // plotter Serial.print(low); Serial.print(","); Serial.print(high); Serial.print(","); Serial.print(mean); Serial.print(","); Serial.print(median); Serial.print(","); Serial.println(redVal); } } // tigoe's sort for an array void bubbleSort() { int out, in, swapper; for (out = 0 ; out < sampleSize; out++) { // outer loop for (in = out; in < (sampleSize - 1); in++) { // inner loop if ( readings[in] > readings[in + 1] ) { // out of order? // swap them: swapper = readings[in]; readings [in] = readings[in + 1]; readings[in + 1] = swapper; } } } }
Class notes, 22 Oct 2020
Great posts about how you react to sound. This is what we’re looking to do with sound — bypass thinking about what you’re hearing and have an immediate reaction.
Sound crit moved to 10 Nov. 3 Nov will be a workday in A10, 5 Nov will be another workday-not-in-A10. I’ll be on zoom for both classes.
Notes
We’re looking at sounds-out-of-context. Experimental music, the use of new technology to invent new music, etc.
Experimental / avant-garde sound
Mark Applebaum’s experimental instruments and scoring. https://www.youtube.com/watch?v=46w99bZ3W_M
Nikoli Voinov who composed music by drawing on paper, creating animation that made sound. https://www.youtube.com/watch?v=Z7Zb4rso82M
Musique concrete using early technology to record and modify sound, including the original soundtrack to “Doctor Who”. Some examples of recording found sound and reusing it for music. https://www.youtube.com/watch?v=c4ea0sBrw6M
Some music you should listen to as background music while you’re doing other tasks. Avant-garde and futurism is a rather wide grouping, like saying “rock” or “country”:
Experimental composition with traditional instruments https://www.youtube.com/watch?v=TTUXhLeLojA including John Cage: https://www.youtube.com/watch?v=pq0a317mk30
Luigi Russolo’s “industrial music” using sirens https://www.youtube.com/watch?v=BYPXAo1cOA4
The Development of Hip-Hop
Using street technology to change and create new genres of music. Entertainment and environmental sounds can come from other contexts with the use of equipment to record, store, modify, and replay.
Turntables used to create hip hop https://www.youtube.com/watch?v=oA-OpvH4CIQ and the 1.5 hour documentary (good for a long flight) https://www.youtube.com/watch?v=YEKRAn-ZleM
Appropriating Sounds from Other Domains
There are a couple of famous sounds in hip-hop that became parts of pop and rave music.
Watch these:
- The Orchestra Hit https://www.youtube.com/watch?v=8A1Aj1_EF9Y
- The Amen Break https://www.youtube.com/watch?v=5SaFTm2bcac
Save this for free time:
Rob Base teaching DJ at the university level https://www.youtube.com/watch?v=spto6rbvvV0
Starting interaction with sound, 20 Oct
Why is sound important?
Why is sound important? How do you close your earlids when you go to sleep?
How do we learn to make sounds as children? https://www.youtube.com/watch?v=8fHi36dvTdE&feature=youtu.be&t=920
Close your eyes after following links in this section. don’t worry about the visual details and information, this is learning to understand sound and signals
Classes of sounds
Signals and alerts
Signals and alerts are short sounds that transfer a small amount of information
- automobile warning buzzers and interior sounds that communicate the state of the vehicle: https://www.soundsnap.com/tags/car_interiors
- walk/don’t walk beep boop and how crazy we were about it at CMU: https://www.youtube.com/watch?v=dcE6gn-JLbw
- fire department / police codes on dispatch channels: http://www.policeinterceptor.com/emerg.htm
- collection that includes a game tone https://www.zedge.net/find/ringtones/dispatch
- doorbell: https://www.youtube.com/watch?v=sbCyxgLgmEE https://www.youtube.com/watch?v=fUVj0BnC8Gw
- ringing phone: https://www.youtube.com/watch?v=3GnoCz5mw3M
Information over time – songs and patterns, more complex information, many of them skeuomorphic
- air raid siren, dual pitch: https://www.youtube.com/watch?v=cURcd2_w-rg
- tornado sirens: https://www.youtube.com/watch?v=Djqi86jMl5g
- my borough uses this as the 15 minute warning on curfew for minors(!)
Music and entertainment
Ringing phones are so unique that we map and learn our own ringtones using songs. Golan Levin’s mobile phone concert was only possible because phones had ring tones that couldn’t be changed. http://www.flong.com/projects/telesymphony/
Star Trek had one of the earliest catalogs of special effects sounds used to alert viewers of plot elements and activity. https://www.mediacollege.com/downloads/sound-effects/star-trek/tos/
Professional companies that sell sound libraries. https://www.soundsnap.com/
Chili’s used the sound of sizzling fajitas to sell them, not their flavor: https://99percentinvisible.org/episode/the-sizzle/
We like things that reflect heartbeat rhythm, this goes back to dance for thousands of years, continue to do this in today’s music with multiple tempo compositions. Orbital and Underworld are two electronic bands using dual internal tempos, one near the rate of resting heart beat the other near the rate of active (dancing) heart beat. https://www.youtube.com/watch?v=IcQXy4YdFcM
Psychological effects of sound
Is it genetics that cause us to respond to the sound of a crying human baby? Can you think of an “angry” noise? A “happy” noise? a “relaxing” noise?
Arduino source formatting tool
I’ve added a tool that shows up in the Post editor as the icon “{…}”. To post the code below, I clicked on that icon, pasted in my code, turned off indentation, then selected C++, and hit “OK”.
// -*-c++-*- /* The MIT License (MIT) Copyright (c) 2017 J. Eric Townsend Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. */ /* using interrupts to stop/alter a process that's eating up the loop() cycle with delays */ static const int potOnPin = A1; static const int potOffPin = A1; static const int togglePin = 2; static const int LED1Pin = 5; static const int LED2Pin = 6; static const int LED3Pin = 9; int LEDOnDuration = 1000; // milliseconds int LEDOffDuration = 1000; // milliseconds long LEDOnTime = 0; long LEDOffTime = 0;; bool LEDState = false; bool LEDStatePrev = false; int toggleState = 0; bool flagState = false; const bool isInterrupt = true; // TODO: how do we stop the for() loops so we can do another analogRead()? void SwitchPressed() { // we can set the delay to 0 and fall out of the loops //LEDOnDuration = 0; //LEDOffDuration = 0; // or we can modify a flag variable and use that to exit the loop flagState = true; } void setup() { Serial.begin(115200); pinMode(potOnPin, INPUT); pinMode(potOffPin, INPUT); pinMode(togglePin, INPUT); pinMode(LED1Pin, OUTPUT); pinMode(LED2Pin, OUTPUT); pinMode(LED3Pin, OUTPUT); if (isInterrupt) { attachInterrupt (digitalPinToInterrupt(togglePin), SwitchPressed, CHANGE); } } void loop() { if (!isInterrupt) { flagState = digitalRead(togglePin); } if (flagState) { Serial.println("flagState true"); } // take input from the pots to determine on/off times int pot = analogRead(potOnPin); if (pot != LEDOnDuration) { //Serial.print("on: "); //Serial.println(pot); LEDOnDuration = map(pot, 0, 1024, 0, 255); } pot = analogRead(potOffPin); if (pot != LEDOffDuration) { // Serial.print("off: "); // Serial.println(pot); LEDOffDuration = map(pot, 0, 1024, 0, 255); } // now let's write some really bad code // turn the LEDs on for (int i = 0; i < 255; i++) { if (flagState) { break; // is more reliable than setting i = 256; // if we set flagState back to false here we // won't make a change in the next loop where we // turn LEDs off } else { analogWrite(LED1Pin, i); analogWrite(LED2Pin, i); analogWrite(LED3Pin, i); delay(LEDOnDuration); } } // turn the LEDs off for (int i = 255; i > 0; i--) { if (flagState) { break; // is more reliable than setting i = 256; // if we set flagState back to false here we // won't make a change in the next loop where we // turn LEDs on } else { analogWrite(LED1Pin, i); analogWrite(LED2Pin, i); analogWrite(LED3Pin, i); delay(LEDOffDuration); } } // this is the proper place to turn off the flagState. // We could do this with just a statement and never check // its status, but using a test makes debugging easier if (flagState == true) { flagState = false; // because we could add this Serial.println("flagState was on, now turned off"); } }
Class Notes, 8 Oct 2020
Kinetic input devices
In 1968, Doug Englebart demonstrated the first “workstation”. It’s a long watch but I think at least the first half will give you a lot of ideas on how to pitch a novel technological concept. Stanford has a broken down version in flash.
Doug Engelbart’s 1968 Demo in full from Frode Hegland on Vimeo.
Accessibility vs Inclusion
What makes something accessible? Is universal design also accessibility?
Inclusion ==> inviting, making someone want to participate. How do you invite someone to provide input / direction?
Microsoft has a great program on
Microsoft Inclusion and PDF book
Are 30mm arcade buttons are accessible? Interrupt or constant? Convex or concave? If you want to use Universal Design, how do you decide how big the button should be and where it’s located?
There’s a wide variety of arcade push buttons. Are controls like buttons the wrong answer? Is a better way to collect input?
What is wrong with the E-Stop button in A10?
- Unlit
- Recessed button “hidden” in a guard
- No signage on the wall like we have with fire extinguishers
Kinetic Output and representation of data
tactile maps: https://www.youtube.com/watch?v=psObymNkzvk
research data on tactile map comparison: https://www.youtube.com/watch?v=Qbe9G9dWcSk
tactile graphics using “swell paper”: https://www.youtube.com/watch?v=QeulfaWn_Ps
what does a snowflake look like? A butterfly? A sailboat?
3d printing for the blind: https://www.youtube.com/watch?v=bEVTu1xdpVI
Not discussed in class — make your own 3d prints of maps. https://www.instructables.com/Make-3d-Printed-Topo-Maps-of-Anywhere/
Interrupts and kinetic interaction
- Interrupts as inputs to state machines
- Human interruption as part of a UI
- Environmental interruption as part of communication. How do we interrupt each other (politely!) during conference calls or video sessions?
- Interrupts that limit motion — end-stop switches
ASL interpreters and physical communication
In lecture I was talking about ASL translators for public speakers, but I think interpretations of music and lyrics are even more dramatic
Cardi B, Bodak Yellow in ASL
Hamilton, “Wait for it“, “Guns and Ships“, and “Alexander Hamilton” by a group with nice costumes.