Class Notes: 7 November, 2019

Playing catch-up on a variety of topics.

Not discussed in class: hiding things in plain sight sometimes using asshat mobile phone “security”.  Any physical penetration tester could find/defeat these within minutes but a drunk person at a party would not do so well.  John Wick had a good plan.

Foley artists creating the sounds of weather.

The Arduino guide to handling debounce on switches.

A mapping of western scale notes to frequencies.

The Sound Noise Device that uses cameras to create animation and sound.

It’s easy to create sounds without speakers:


Class Notes: 4 November, 2019

Sound sources

Please watch the shorter ones, I linked to a couple of > 1 hour sources for future reference.

Experimental / avant-garde sound

Mark Applebaum’s experimental instruments and scoring.

Nikoli Voinov who composed music by drawing on paper, creating animation that made sound.

Musique concrete using early technology to record and modify sound, including the original soundtrack to “Doctor Who”.  Some great examples of recording found sound and reusing it for music.

Avant-garde and futurism is a rather wide grouping, like saying “rock” or “country”:

Using street technology to change and create new genres of music.  Entertainment and environmental sounds can come from other contexts with the use of equipment to record, store, modify, and replay.

Turntables used to create hip hop and the 1.5 hour documentary.

The Orchestral Hit

The Amen Break

Genre conflation

This is a counter to sampling and using electronics to invent new music.  Instruments from one style of music are used to perform a style of music from a completely unrelated genre.  My favorite example is “pirate metal” or the band Orkestra Obsolete playing famous pop music.

Class notes: 31 October, 2019

Starting Sound

Why is sound important?  We have binaural hearing and can point to the direction of a sound without any practice.

How do you “close your earlids” when you go to sleep?  How do children learn to speak?   Leonard Bernstein’s experiments with making sounds like a child and how common those sounds might be across cultures.

Close your eyes after following links in this section. don’t worry about the visual details and information, this is learning to understand sound and signals.

Classes of sounds (one view)

Signals / alerts — short sounds that transfer information

Songs and patterns that transfer information over time

We have a history of using air raid sirens from WWI and WWII as a means to notify the population of an area of an event or condition.

  • air raid siren, dual pitch  — my borough uses one of these at 9:45pm to signal curfew hours for children.  The local volunteer fire department have a different one they use to alert volunteers to a fire.
  • tornado sirens used to notify a town/area of a possible tornado

Music and entertainment

Psychological effects of sound

Is it genetics that cause us to respond to the sound of a crying human baby?  Can you think of an “angry” noise?  A “happy” noise? a “relaxing” noise?

Sketches, Fritzing, and 3d models

Sketches from sound-class-1 (including one I DID NOT show in class) and Fritzing a speaker to a transistor.

3d printed air raid sirens.

Class notes: 22 October, 2019

Kinetic Input/Output

The Demo”, showing off the mouse, chord keyboard, and social media.

Accessibility / Inclusion

Microsoft’s Inclusion and a PDF copy of their book.

What is accessible?

Are 30mm arcade buttons are accessible? Do they simply nterrupt or do they provide constant state?  Are the buttons convex or concave?  How high are the guards around the buttons?  If you want to use Universal Design, how do you decide how big the button should be and where it’s located?

What is wrong with the E-Stop button in A10?

  • Unlit
  • Recessed button “hidden” in a guard
  • No signage on the wall like we have with fire extinguishers

Are controls like buttons the wrong answer?  Is better output the way to go?

tactile maps

presentation of research data on tactile map comparison

tactile graphics using “swell paper”

3d printing reference objects for the blind — what does a snowflake look like?  A butterfly?  A sailboat?


Kinetic crit on 29 Oct.  Thursday office hours + Thursday class is a work day.

Class Notes: 10 October, 2019

Input classification, serial communication, interrupts

Types of Input

monophonic + skill: wind instruments, percussion

polyphonic + tech: keyboards, pianos, organs, strings
anthropomorphic: respond to human condition, heart rate, blood pressure, galvanic skin, breath rate, pulse rate, visual interpretation of secondary movements: eye twitch, touching your face, blinking

Golan Levin “Opto-Isolator

Serial Communication

SPI/I2C and complex communications protocols
How we get complex data from sensors – a lot of this is hidden in libraries
Unique IDs
Simple controls for complex output: neopixel
SparkFun’s version: Qwiic


Show examples of interrupt code in the environment
switches on mobiles
remote controls for the projectors
complex interrupt systems in video game controllers
rotary encoder (we’ll do a demo later in the semester)
for now, we only use digital inputs for interrupts

Code samples, show how an interrupt can be used to toggle a state by one increment compared to holding down a switch and falling through a number of states.
Note that holding down the switch means the interrupt service routine (ISR) only functions once
Compare to using delay() to sample data every so many units of time.

Use an interrupt to stop a task that takes a long time, say a long for() or while() loop, by adjusting the terminating conditions

Question: What if you were playing mp3 files or video, how would you use interrupts as part of the interface?

zip file with some example interrupts

Class Notes: 8 October, 2019

Crits will move to Tuesdays

Haptic/touch vs.  objects moving in space

touching a person vs. moving an object – touch is more personal, requires little energy.   Touch can be wrapped in a robotic device, ex: Paro (wiki) trade show demo.

Moving things typically requires an external power source, Arduino can only provide 5V at a few milliamps.

For moving things, use basic physics

  • gears — FAQ on 3d printing gears
  • levers — drive a lever with a solenoid, transfer short movement to long
  • pulleys – drive with DC motors or steppers

Class notes: 3 October, 2019

Physical representation of information

For the purposes of this class we’re looking at physical representation of information over time at a small scale.  Think tabletop or handheld representations.

Some examples:

  • vibration: usually done with “tactors” but also a feature of mobile devices and handheld devices.  Mobile phones can vibrate, some tools for electronics vibrate as a way of sending notification.
  • thumps and pokes:  using motion from a solenoid or servo to relay information with pressure or tapping motions
  • temperature: peltier pads (what we use to cool CPUs and GPUs) that can heat/cool, flowing water or air that is heated or cooled.  Electric heaters or coolers are probably too complex for this class.  Dry ice is another option but can be hazardous to work with.
  • symbols: Braille terminals

Adam Savage’s DIY costume cooling vest for cosplay and a commercial alternative.

Physical representation of information over time

Using motion over time

  • signal encodings of language: Morse code
  • pattern recognition: what motion feels like walking? Running? Being happy or sad?
  • meaning is generated by content that changes over time
  • School for Poetic Computation

Coaching vs. grading

Think  about coaching, providing good feedback and encouragement to take a positive action.

Example: sports trainer that monitors your HR, BP, breathing rate, and hydration and knows your training course.  It encourages you to do better instead of punishing you for not doing enough.

Example: music “coach” that helps you learn to perform music. Watches your body and helps you correct form/posture.  Reminds you that you are always performing, even when you’re just practicing a scale or an etude.

Alice Miller’s “For Your Own Good“, a criticism arguing that we replace the pedagogy of punishment  with support for learning, using the German pedagogy that gave rise to support of fascism as one study.

Class notes: 1 October, 2019

Kinetics 1

Consider the size of physical control related to other controls and the context for the controls.

Is there physical feedback that you’ve used a control or do you have to look at a screen to know the effects of your control change?

Is physical feedback encoded — one buzz is a phone call, two buzzes is a text message — or is it just an alert to request that you look at a screen?

Controls can be stylistic or even skeumorphic: cars that look like airplanes vs. boxy economy cars.

We fly spacecraft with computers, there are no joysticks ala Star Wars or any other movie that uses WWII airplane controls to navigate in space.   However, on the Enterprise…

MIX MECHANICAL AND OTHER CONTROLS WHERE APPROPRIATE Mechanical controls are better for some uses, though they can’t as easily serve multiple functions. Nonmechanical controls, like touch-screen buttons, are easier to change into other controls but don’t offer the same kind of haptic feedback, making them impossible to identify without looking at them and creating questions about whether they’ve been actuated. Design interfaces with an appropriate combination that best fits the various uses and characteristics.

– Shedroff, Nathan. Make It So: Interaction Design Lessons from Science Fiction (p. 26). Rosenfeld Media. Kindle Edition.

Data filtering and cleanup

look at input over time for kinetic outputs

How do you smooth data, similar to what I showed on the whiteboard?

Reading assignment

Arudino tutotorial on simple smoothing

20 minute tutorial on smoothing analog input

Find some examples of data over time that you can interact with, not just respond to.  ex: weather forecast changes your todo list so your outdoor errands happen when it’s not raining.

Re: Story of Your Life, etc…

Arrival is perhaps my favorite film, and while I was already familiar with the Wolfram documentary, and the original Ted Chiang story, I am always excited to revisit these ideas, and learn more about the interplay of language and cognition. I was impressed with the way the Electric Didact dissected this concept in the film and tied it back to the root of the very word “understand.”

Even more interesting to me is when we try to use language to express what we see, thereby translating visual cognition into an audible expression and back again. (As we are aiming to do with our projects in this course.)

This idea reminds me of a study that found that Russian speakers, who have separate words to distinguish between light and dark blue, are quicker to recognize these subtle differences than English speakers when shown two different shades, thus indicating language affecting visual perception right here on our own blue planet.

Article from the National Academy of Science

On the other end of the same cycle, Vox did an interesting piece looking at the evolution of words for color in language across different cultures, beginning almost always with just light and dark, then next to red before blue and green.

I’m curious if there is any way to actively adapt the interconnection between visual and linguistic cognition for use in interface design, or to create new connections by building a new vocabulary to map optical cues to concepts that do not have representations in the visual spectrum.