Author: Jet Townsend

Class notes: 3 October, 2019

Physical representation of information

For the purposes of this class we’re looking at physical representation of information over time at a small scale.  Think tabletop or handheld representations.

Some examples:

  • vibration: usually done with “tactors” but also a feature of mobile devices and handheld devices.  Mobile phones can vibrate, some tools for electronics vibrate as a way of sending notification.
  • thumps and pokes:  using motion from a solenoid or servo to relay information with pressure or tapping motions
  • temperature: peltier pads (what we use to cool CPUs and GPUs) that can heat/cool, flowing water or air that is heated or cooled.  Electric heaters or coolers are probably too complex for this class.  Dry ice is another option but can be hazardous to work with.
  • symbols: Braille terminals

Adam Savage’s DIY costume cooling vest for cosplay and a commercial alternative.

Physical representation of information over time

Using motion over time

  • signal encodings of language: Morse code
  • pattern recognition: what motion feels like walking? Running? Being happy or sad?
  • meaning is generated by content that changes over time
  • School for Poetic Computation

Coaching vs. grading

Think  about coaching, providing good feedback and encouragement to take a positive action.

Example: sports trainer that monitors your HR, BP, breathing rate, and hydration and knows your training course.  It encourages you to do better instead of punishing you for not doing enough.

Example: music “coach” that helps you learn to perform music. Watches your body and helps you correct form/posture.  Reminds you that you are always performing, even when you’re just practicing a scale or an etude.

Alice Miller’s “For Your Own Good“, a criticism arguing that we replace the pedagogy of punishment  with support for learning, using the German pedagogy that gave rise to support of fascism as one study.

Class notes: 1 October, 2019

Kinetics 1

Consider the size of physical control related to other controls and the context for the controls.

Is there physical feedback that you’ve used a control or do you have to look at a screen to know the effects of your control change?

Is physical feedback encoded — one buzz is a phone call, two buzzes is a text message — or is it just an alert to request that you look at a screen?

Controls can be stylistic or even skeumorphic: cars that look like airplanes vs. boxy economy cars.

We fly spacecraft with computers, there are no joysticks ala Star Wars or any other movie that uses WWII airplane controls to navigate in space.   However, on the Enterprise…

MIX MECHANICAL AND OTHER CONTROLS WHERE APPROPRIATE Mechanical controls are better for some uses, though they can’t as easily serve multiple functions. Nonmechanical controls, like touch-screen buttons, are easier to change into other controls but don’t offer the same kind of haptic feedback, making them impossible to identify without looking at them and creating questions about whether they’ve been actuated. Design interfaces with an appropriate combination that best fits the various uses and characteristics.

– Shedroff, Nathan. Make It So: Interaction Design Lessons from Science Fiction (p. 26). Rosenfeld Media. Kindle Edition.

Data filtering and cleanup

look at input over time for kinetic outputs

How do you smooth data, similar to what I showed on the whiteboard?

Reading assignment

Arudino tutotorial on simple smoothing

20 minute tutorial on smoothing analog input

Find some examples of data over time that you can interact with, not just respond to.  ex: weather forecast changes your todo list so your outdoor errands happen when it’s not raining.

Interactive Feedback Overload

NTSB cites competing pilot warnings and flawed safety assumptions on Boeing 737 Max

After an automated feature on a Boeing 737 Max failed in the skies above Ethiopia in March, repeatedly forcing the plane’s nose downward, the pilots were bombarded with a cacophony of alarms that shook, clacked and lit up throughout the cockpit.

Boeing did not sufficiently consider the effect that such a barrage would have on those flying the plane when it designed the Max, according to the National Transportation Safety Board, which released its first wave of recommendations Thursday in response to the crash in Ethiopia and one in Indonesia under similar circumstances in October.
[...]

Critique 1: Visual Interaction

“An island that cannot hear in an ocean that cannot see.”

Due 11:59pm, 25 September.

Use vision to make an interaction accessible to someone that cannot hear.   This is more than the simple state machines in the weekly assignments — for this crit we want a full interactive experience where the device interacts with a person.  “Cannot hear” is not defined only as deaf or hard of hearing, it can be a condition where listening/hearing information is impossible.  Examples:  at a Baroque symphony, wearing ear protection while using loud construction equipment like a jackhammer, at night in a dorm room when the roommate is sleeping.

The inputs that can be used for this interaction are open to whatever makes the interaction work.

Take a look at the syllabus for more information on crits and the goals of this class before starting your project.  Remember what we talked about in the first classes, the differences between reactions and interactions.  The IDeATe Lending Library is also a good resource (and some of the staff have taken Making Things Interactive!).

Email me if you have any questions or hardware problems.  I will be on the road most of Thursday but will have my laptop out at the conference taking notes on presentations.

Class Notes: 19 September, 2019, reading and listening assignments

Story of Your Life, by Ted Chiang.  A short story about alien language told only using the alphabet.   The movie based on this story, “Arrival”, was the visual version of a foreign alphabet “Armied up” so it has more tension.  Christopher Wolfram shows how he created the language for the movie with links to his code (it’s long and probably boring if you don’t think in Mathematica).  A much shorter, more philosophical question about the meaning of language uses “Arrival” as its base.

Two listening pieces, one based on “Deafspace” architecture, the other on wayfinding (which we really haven’t discussed in class yet)

Deafspace

Walk This Way

 

 

 

Class Notes: 17 September, 2019

Visualization of Sound and Language

Visualization of sound by audio frequencies

Visualization of simple tones

Visualizing music in real time using sound

Visualizing music using sheet music

Posters visualizing songs based on the sheet music for the song.

Bach’s Cello Suite No. 1 as a poster, as sheet music, and performed by Yo-Yo Ma.  Compare the visualization of the poster with the instructions in the sheet music.

Visualization of language

Visual ASL dictionary with full demonstrations of each word.

Written ASL (which we did not discuss in class), a notation system of translating ASL hand motions to marks on paper.

Compare ASL visualization of words to Braille‘s visualization of letters and word components.  Note that the shapes of the letters in Braille do not map to the shape of the letters used in print.

Assignment 4: Communicate Changes in State

Use visual information to communicate a change in state

We discussed examples in class to show how a visual indicator can show a change in state.  My microwave and washing mashing both make noises when they have completed an action and are ready for me to respond.  How could I be visually notified of a state change if I had hearing limitations; or if I were on the wrong floor and couldn’t hear the notifications with normal hearing?

Class notes: 12 September, 2019

State machine transitions

Documenting a state machine: Omnigraffle (mac) vs. SmartDraw (win10) vs. ??? (linux) vs. whiteboard.  One nice thing about mobile phones is that you can now do “save as” on a whiteboard simply by taking a picture.

Finite state machine

A finite state machine (FSM) needs states, transitions, and actions which are transitions that operate outside of the state machine.  We use the word “finite” to define that there are only a certain number of states in a machine and that a machine can only be in a single state.

This is a valid FSM:

automobile_door:  open_unlocked, closed_locked, closed_unlocked

This is an invalid FSM:

automobile_door:  open, closed, locked, unlocked

Ask yourself why one is valid and how the other could be invalid.

Multiple FSMs

Say you have two state machines for two actors in a game, “Barney” and “The Monster”.  Each will have it’s own set of states, but a change in one state — The Monster goes to “is visible” — sends a signal to Barney to changed to the state  “on_patrol” so it is looking for The Monster.

Where do Barney and The Monster exist?  They could be child state machines of The Encounter Room which has states of lights_on, lights_off, emergency_alert, and fire_sprinklers_activated.   Barney and The Monster can have transitions that also notify The Encounter Room and allow it to make decisions on changing it’s own state or sending state change mechanisms to Barney and The Monster.

How can we define an “action” and not a “transition”?

Movie transitions that also add to the plot:

Alerts of state changes that let you know an otherwise undetectable FSM has changed states:

  • microwave ding when it’s finished heating
  • countdown timer to start an event (from waiting -> running)
  • RFID EZPass validation light
  • elevator alerts for current floor and direction

When can visual interaction replace sound or motion?

  • baby monitor that translates sound to video
  • GFCI lights that indicate status of interrupt
  • replace sound warning with video flash, Mac Terminal

What sounds are important in FSM and what sounds are simply decorations?

Turning keypress sounds on and off on phones and keyboards

Dialer tones (DTMF) in response to pressing buttons on a mobile that doesn’t use DTMF

a range of car horns for different listeners

faked car sounds to impress the driver and passengers