Making Things Interactive, Fall 2020
Making Things Interactive
If you’ve never dealt with input smoothing, this 20 minute tutorial on smoothing analog input on arduino covers all the basics
Assignment #5: describe translating a sound or image in to a mechanical output, assume it’s for accessibility. No need to make anything, this is a thinking/drawing exercise that should take no more than an hour
If you find some examples of data over time to interact with (hint: thursday assignment) please post to Looking Outward. (no data for stock markets nor weather)
A10 dates updated on blog
reminder, campus is closed after Thanksgiving
looking for parts, try octopart search engine: https://octopart.com/
do people want 3d prints or lasercuts or both?
size of physical control vs. input
size of physical control vs. output
tactile controls are great for fine control/refinement/detailed feedback
but can get lost in the shuffle — NASA controls are laid out based on the physical design of the space shuttle
or can be stylistic / skeumorphic. Why does a Starfleet vessel have touchscreens (LCARS) everywhere but the warp engines are driven by an 19th century ship’s throttle?
MIX MECHANICAL AND OTHER CONTROLS WHERE APPROPRIATE Mechanical controls are better for some uses, though they can’t as easily serve multiple functions. Nonmechanical controls, like touch-screen buttons, are easier to change into other controls but don’t offer the same kind of haptic feedback, making them impossible to identify without looking at them and creating questions about whether they’ve been actuated. Design interfaces with an appropriate combination that best fits the various uses and characteristics.
median vs. mean
std devs see wiki for details: https://en.wikipedia.org/wiki/Standard_deviation
how to read complex sensors over serial protocols: I2C, SPI, MIDI
look at data smoothing / filtering
simple smoothing: https://www.arduino.cc/en/Tutorial/Smoothing
break down the types of kinetic interaction
focusing on output
– vibration
– thumps, pokes
– temperature? peltier boards
– symbols: Braille
Then over time
– signal encodings, morse
– pattern recognition: what does walk feel like? Run? Crying? Laughing?
– earthquake pattern recognition
– meaning generated by content that changes over time, poetry
Assignment 4: Start investigating kinect input and output
Reading assignment from Make It So
Find some examples of kinetic interaction — not reaction — and post to looking out
Email me with how you feel about crits on Ttu if you can use A10 on Tue
Goal: Use vision to make something accessible to someone without hearing or manual sense of touch. These do not have to be physical disabilities, it could be construction works wearing hearing protection or heavy protective gloves that prevent them from having a sense of physical touch.
Find a problem and solve it. The problem needs to have a context.
I’ve made a request that Tuesday’s work class be held in A10. Will email an update.
dark skies as a source of crowd-sourced weather data, scrape data from https://darksky.net/forecast/40.4346,-79.8655/us12/en
Interfering with sensors goes back to “Dazzle Camouflag”: https://en.wikipedia.org/wiki/Dazzle_camouflage
Virag Varga’s research on devices that communicate with one another through skin: https://dl.acm.org/profile/99658719699
Documenting a state machine: Omnigraffle (mac) vs. SmartDraw (win10) vs. ??? (linux) vs. whiteboard
A finite(explain) state machine needs states, transitions, and actions (transitions that do things)
Example: UI button that changes screen color has a state variable with the current color
state machines in video games, hierarchies of state machines
how do we define a transition from one state to another?
– segue: pan, wipe, blur
– alert:
– microwave ding when it’s finished heating
– countdown timer to start an event (from waiting -> running)
– elevator alerts
what can visual output replace? sound? motion?
babymonitor that translates sound to video
GFCI sound vs. click
replace sound warning with video flash
ubicoustics: http://www.gierad.com/projects/ubicoustics/
how can we replace sound from a state machine with visual queues?
what sounds are important in state machines and what sound are decorations?
keyclicks on/off phone
annoying car noises
car sounds effects
visualizing sound waves
https://vimeo.com/partitura
visualizing music:
Vibrant Data Visualizations of Famous Classical Music Scores Burst with Color
https://www.youtube.com/channel/UCRQH9-hWxELNCv47z2O5nfg
ASL: https://www.handspeak.com/
Braile: http://www.brailleauthority.org/formats/2011manual-web/index.html
Arrival — read “The Story of Your Life”
the movie version: https://www.space.com/35696-arrival-movie-alien-language-explained.html
99pi episodes 50 “Deafspace” (Gaulludet) and 126 “Walk This Way” (wayfinding)
When your senses get crossed, Synesthesia
who needs access to A10? We can only do it Tue night during class hours.
bookstore has not shipped yet. we will get tracking #’s when they do.
This is in the kit: ELEGOO UNO Project Super Starter Kit with Tutorial and UNO R3 Compatible with Arduino IDE
trying to deal with lost class time: an assignment for next tue, then a crit on the 24th that includes the assignment?
what went right?
what went wrong?
The “thing like a pot” is a rotary encoder
finite: what we’re doing in class, fixed states and paths
abstract: can modify itself to adapt to changes in data.
Reactive state machines:
Hourglass timer – set time, start, stop, reset
Arcade game – attract / play / reward
gambling devices slot machines
Murphy bed attached to lights
Interactive state machines:
content predictors on Amazon/Netflix/eBay, “if you like…”
advertising predictors, “if you like xxx we want you to buy yyy”
microwave that can adjust cooking time based on humidity (I have one of these!)
traffic lights with IR detectors for EMS vehicles
Music instrument as state machine:
https://www.humaninstruments.co.uk/instruments
What if state machines were smart(er)? What are interesting problems for them to solve?
What are predictive states machines? console gaming turned in to physical gaming
auto-aiming / target tracking
catch a frisbee (dogs know calculus)
Can made predictions based on history/sensors? Your car notices you’re a sloppy driver (exhausted) and limits your speed. Duolingo detects you’re having vocab problems and changes your exercises (this is an ASM, arguably).
post some interesting state machines to Looking Outward
This is pretty interesting — a haptic baton created by Human Instruments who work on accessibility for musicians.
Crits/projects due on Tuesdays. Reading/listening/watching assignments due Thursdays.
Install Arduino, run test hello world sketch
Install p5.js and serial control: https://github.com/p5-serial/p5.serialcontrol/releases
Dan Schiffman has great tutorials.
The less types and amount of visual feedback you give the better
Ex: a clock needs hours and minutes, but does it need seconds? Microseconds? Centuries?
What do the clocks in my kitchen need to show me? Why do I have so many clocks in appliances that can set an action for a specific time in the future?
Combining the visual and haptic, the Bradley watch.
Did Susan Kare invent emoji? Her work at Apple
led to her work at General Magic, making “stamps” for the cutting-edge Magic Link “PDA”.
how many lights do you have in your kitchen?
Make It So: Forward, Chapter 1, Chapter 3. (Chapter 2 if you have time.)
But first, story telling
Tell a story, not a novel, not an elevator pitch.
A story has at least one actor, a beginning, a plot, and an end.
In interaction design “plot” is the interaction.
What is the plot of a doorbell? Someone visits, they want to get your attention (as you don’t spend all day staring at the door), they push a button, an action happens, and you know someone is at the door.
How do doorbells work for people without hearing? What did we do before electricity?
But first, visual skeuomorphism. A skeuomorph is a object derived from past objects and contians past visual cues that have no meaning in current culture. Ex: the “phone” icon on my phone that looks like the physical handsets we stopped using in the 90s.
is a derivative object that retains ornamental design cues (attributes) from structures that were inherent to the original.[3] Examples include pottery embellished with imitation rivets reminiscent of similar pots made of metal[4] and a software calendar that imitates the appearance of binding on a paper desk calendar. Or the “save” icon that looks like a floppy drive, that a Japanese student thought looked like a drink dispenser: https://twitter.com/fea0er/status/1160099135569063936
When you are creating visual representations, are you using your culture’s history of symbolism?
Console displays in video games – metadata about your status. health, money, power, speed.
Why do some of the games with great graphics still use bar graphs to display raw data? We could hear our speed. Our character/avatar could change color. Why do we just have bar graphs?
What do colors mean? What states do they reflect?
Use visuals to reflect the state of a state machine that normally uses sound to show state. If you have your Arduinos, please use hardware. If not, p5.js or other software on a computer is fine. Show the state machine in your response, a diagram or words are both fine.
If you’re looking for some cues, 99 Percent Invisible has a great, 15 minute podcast on “Deafspace“.
Questions: I rely on my water boiler, microwave, a kitchen timers to beep at me when certain states are changed. What’s a visual mechanism that doesn’t require me to simply stand there and wait? (English expression: “A watched kettle never boils.”)
Due Mon, 14 Sep, 11:59pm.
Reading: from “Make It So“, Forward, Chapter 1, Chapter 3. (Chapter 2 if you have time/interest.) Post your thoughts on the chapters and content.
Counting: Once it’s dark outside, turn off the overhead light in your kitchen (or in your room if you don’t have a kitchen) and count the number of lights. A clock is one light, but if you have status LEDs on electronics, each of those is a light. You don’t need to post this, but have a list to remind you during class.