SPI/I2C and complex communications protocols
How we get complex data from sensors – a lot of this is hidden in libraries
Unique IDs
Simple controls for complex output: neopixel
SparkFun’s version: Qwiic
Interrupts
Show examples of interrupt code in the environment
switches on mobiles
remote controls for the projectors, TVs, stereos
little to no latency
complex interrupt systems in video game controllers
rotary encoder (we’ll do a demo later in the semester)
for now, we only use digital inputs for interrupts
Code samples, show how an interrupt can be used to toggle a state by one increment compared to holding down a switch and falling through a number of states.
Note that holding down the switch means the interrupt service routine (ISR) only functions once
Compare to using delay() to sample data every so many units of time.
Use an interrupt to stop a task that takes a long time, say a long for() or while() loop, by adjusting the terminating conditions
Question: What if you were playing mp3 files or video, how would you use interrupts as part of the interface?
Consider medical/physically safety with devices that touch the skin. It’s really easy to get burned/frozen with Pletier plates and ice. Instead of smoke, use soap bubbles and a fan.
Coaching vs. grading
Think about coaching, providing good feedback and encouragement to take a positive action instead of negative feedback. In Total Control Training we use positive feedback and coaching. Instead of saying, “You did that wrong, you were too slow going in to the corner,do it again” we say “That’s good, now do it again with a little more throttle as you go in to the corner.”
You want your interaction to be one that invites people and makes them want to interact. If you do a good job, people will wait in line to use your interaction: https://vimeo.com/3796361
Example: sports trainer that monitors your HR, BP, breathing rate, and hydration and knows your training course. It encourages you to do better instead of punishing you for not doing enough.
Example: music “coach” that helps you learn to perform music. Watches your body and helps you correct form/posture. Reminds you that you are always performing, even when you’re just practicing a scale or an etude.
Something to read over the break: Alice Miller’s “For Your Own Good“, a criticism arguing that we replace the pedagogy of punishment with support for learning, using the German pedagogy that gave rise to support of fascism as one study.
looking for parts, try octopart search engine: https://octopart.com/
do people want 3d prints or lasercuts or both?
Discuss reading of Make It So chapters
Start Kinetic discussion
size of physical control vs. input
size of physical control vs. output
tactile controls are great for fine control/refinement/detailed feedback
but can get lost in the shuffle — NASA controls are laid out based on the physical design of the space shuttle
or can be stylistic / skeumorphic. Why does a Starfleet vessel have touchscreens (LCARS) everywhere but the warp engines are driven by an 19th century ship’s throttle?
MIX MECHANICAL AND OTHER CONTROLS WHERE APPROPRIATE Mechanical controls are better for some uses, though they can’t as easily serve multiple functions. Nonmechanical controls, like touch-screen buttons, are easier to change into other controls but don’t offer the same kind of haptic feedback, making them impossible to identify without looking at them and creating questions about whether they’ve been actuated. Design interfaces with an appropriate combination that best fits the various uses and characteristics.
look at inputs for kinetic outputs
median vs. mean
std devs see wiki for details: https://en.wikipedia.org/wiki/Standard_deviation
how to read complex sensors over serial protocols: I2C, SPI, MIDI
Can made predictions based on history/sensors? Your car notices you’re a sloppy driver (exhausted) and limits your speed. Duolingo detects you’re having vocab problems and changes your exercises (this is an ASM, arguably).
Visual changes in state
Doppler shift in nature
lives / health in a video game
traffic slowdowns in Waze
intensity of light => time of day
Mini-assignment for Thursday
post some interesting state machines to Looking Outward
led to her work at General Magic, making “stamps” for the cutting-edge Magic Link “PDA”.
Fundamental types of visual state
color
motion
intensity
type of display: led, a light, a screen, a moving object in 3 dimensions, metronome
Complex types of visual state
typeface
language
icons
images
Reading assignment
how many lights do you have in your kitchen?
Make It So: Forward, Chapter 1, Chapter 3. (Chapter 2 if you have time.)
Displaying State
But first, story telling
Tell a story, not a novel, not an elevator pitch.
A story has at least one actor, a beginning, a plot, and an end.
In interaction design “plot” is the interaction.
What is the plot of a doorbell? Someone visits, they want to get your attention (as you don’t spend all day staring at the door), they push a button, an action happens, and you know someone is at the door.
But first, visual skeuomorphism. A skeuomorph is a object derived from past objects and contians past visual cues that have no meaning in current culture. Ex: the “phone” icon on my phone that looks like the physical handsets we stopped using in the 90s.
is a derivative object that retains ornamental design cues (attributes) from structures that were inherent to the original.[3] Examples include pottery embellished with imitation rivets reminiscent of similar pots made of metal[4] and a software calendar that imitates the appearance of binding on a paper desk calendar. Or the “save” icon that looks like a floppy drive, that a Japanese student thought looked like a drink dispenser: https://twitter.com/fea0er/status/1160099135569063936
When you are creating visual representations, are you using your culture’s history of symbolism?
States visualization
Console displays in video games – metadata about your status. health, money, power, speed.
Why do some of the games with great graphics still use bar graphs to display raw data? We could hear our speed. Our character/avatar could change color. Why do we just have bar graphs?
colors important because they show wealth: purple for royalty, gold, dichromatic glass. Purple as an expensive color to make so only the rich/powerful could afford clothes dyed purple. (“The Secret Lives of Color“, “Why are Jeans Blue?“
color of money — bills with different colors are easier to sort/find
color of food, white is good/clean. In the south we have “dirty rice” where we mix leftovers with fresh, “clean” rice.
Introduce ourselves and describe what we want out of the class
Class blog — I have to “promote” you after you sign on for the first time.
A10 cannot be used outside of class time. I’ll see if I can create a “class time” for MTI
kits are available: https://bookstore.web.cmu.edu/SiteText?id=73598
How do we handle the end of the semester?
Pick My Office Hours
Discuss
revisit tigoe’s page — what was physical interaction and what was reaction? Do you see any themes in projects that aren’t allowed? is there something you’re tired of seeing?
reading assignment — what makes something interactive?
does interaction require emotion or just the appearance of emotion? When do we go from “robot” to “uncanny valley” to “human”?
Musicians and DJs working a crowd — who is interacting and who is driving the interaction?
so what is physical interaction? When/where did it start?
Physical Interaction History
Beekeepers – explain the complexity of a hive and how we’re only just now (past 20 years) discovering how bees vote to make hive-wide decisions. “Honeybee Democracy” is a great introduction to how science works and how to write about science: https://amzn.to/2Z6Y9yL
Alchemists – trying to make things happen with substances they don’t understand. If you don’t know about elements and that lead and gold are elements, what decisions are making to interact with substances? People knew that honey bee “propolis” helped prevent infection thousands of years before we discovered bacteria. Sometimes working in ignorance still leads to progress.
New music and dance styles based on sampling and analog synths: Moog, ARP 2600, hip-hop. Early hip-hop was limited to record players and mics; early synths were insanely expensive and shared at fancy studios. Grand Wizard Theodore *invented* scratching: https://www.youtube.com/watch?v=eHCOIK_fICU
Console games with physical feedback, Rez Trance Vibrator: https://www.theverge.com/2016/9/25/13046770/rez-trance-vibrator-ps2
Dance Dance Revolution from arcades to console games
Driving assistance with vehicles: AI? Interaction? What if the car won’t let me turn, start, or stop? How can I disable safety features?
Flight assistance software in commercial aircraft. Is autopilot interactive?
Flight assistance for military aircraft: self-guiding drones, incoming missile warnings for helicopter pilots. Interaction or reaction?
Near-future Interaction
the focus of this class – we’re prototyping for five years out
interacting with intelligent systems that we don’t completely understand and that can make decisions against our will or with results we don’t appreciate
car that thinks you’re too intoxicated, sleepy, or incompetent to drive, so it refuses to move
home automation system that won’t open the doors and let you leave because the particulate count in the air is hazardous and you’re not wearing a mask
police equipment that won’t let you fire at unarmed civilians — pistol trapped in a holster, safety wont go off.
fire engines that won’t engage fires that cannot be contained
entertainment systems that can filter content as part of mental-health
We’re giving “agency” to interactive devices: https://en.wikipedia.org/wiki/Agency_(philosophy)
“Near-future” is a popular base of pop-culture: MCU set a few years from now
Blade Runner 2049 is a bit farther in the future.
practical effects used as input devices to imagined systems of the future
Adam Savage gets a no-spoilers tour of the Blade Runner 2049 prop shop.
show Adam Savage with prop manager, start around 4:12, skip to 14:30, skip to 18:30
This is not new, in the past we talked about the future and technology. “Design for Dreaming”, released in 1956, is all about the future. It’s also all about the sexism in the 1950s, so you can skip to these points, to get the important points: start at 3:30; skip to 6; skip to 8
Good drama is about storytelling — what if interactive devices are characters in the story?
Colossus: The Forbin Project (1970) — our super computer meets the Soviet supercomputer and they plan our future
2001 — HAL 9000, the ship’s computer that decides it does not want to follow the plan laid out by humans
War Games (1983) — A computer controls all of America’s nuclear weapons. Would you like to play a game? the only way to win is to not play the game.
Alien — MOTHER, a semi-intelligent computer that controls spacecraft while humans are in cryosleep.
ST:TNG’s Data — a walking mobile phone smarter than spacecraft computers?
(Terminator movies don’t count — killing spree robots, not interaction)
Farscape — Moya is semi-intelligent spacecraft used by the main characters, and can refuse orders or do things because of her own desire, not that of her passengers.
Assignment 2, due at 11:59 the day before class, so these are due Monday, 7 September
First, order your kit https://bookstore.web.cmu.edu/SiteText?id=73598
This is a pen-and-paper assignment to go over posting to the class blog and what a good submission looks like. If you want to make something with hardware, that’s fine, but it’s not required. Science fiction is ok, fantasy is not. ever. NO UNICORNS.
1.) A smart doorbell, similar to the smart thermostat I talked about on Tuesday, or
2.) Make a useful practical effect by giving interaction to a thing that doesn’t currently interact with humans. For inspiration, watch Blade Runner 2049. Yes, it’s 3 hours long but OMFG. Look at the physical interfaces between intelligent systems, replicants, humans, AI, and agency for androids.
Prepare the presentation as you would for a crit. Look at previous classes for inspiration
No need to leave video on unless we’re having a group discussion so we can raise hands.
Most lectures (starting next week) will be recorded and take roughly half of the class time.
Classroom discussions will not be recorded.
Take-home kits will be available sometime next week. We’ll let you know when they can be ordered, please don’t call the bookstore.
Projects will be turned in on the blog and demoed via Zoom. Live or pre-recorded demos?
Office hours as needed — email me if you’re stuck and want to have a 1:1 Skype
How do you feel about Canvas?
Undergraduate Requirements
Undergrads have a pre-req of Intro to Physical Computing. If you haven’t taken the pre-req but think you know enough to keep up with MTI assignments, stay for this intro then talk to me after class.
Graduate students
Comfortable working with Arduino, equivalent knowledge of Intro to Phys Comp.
Skills needed by all students
Arduino
Arduino IDE in C++ or CircuitPython. VSC is also an option if you prefer.
for() vs. while() loops
arrays (C++)
switch/case
analog vs digital voltages
dictionaries (Python)
integer vs. float (C++)
interrupts
Understand how/why to use state machines
Physical Computing skills
Analog vs. digital electronics
External power — wall outlets and batteries and power supplies
The difference between servo motors, regular motor, solenoids
Can do more than cut-and-paste a sketch and change a few variable names
Can read/create basic sketches in Fritzing
Other Useful Skills
p5.js (javascript) can be used in this class
Python on Arduino is allowable as a substitute for C++, but I can’t help you debug
BeagleBone Black and Raspberry Pi may be used as a substitute for Arduino, again I can’t help you debug
If you want to learn this sort of technology but can’t take this class, feel free to email me for some pointers on how to get started on your own time. There are some “Arduino 101” kits that you could do over winter break that I would count for PhysComp in the future.
Introduction to the Class / go over Administration document
We’ll do real introductions on Thursday
Originally a required class for a MTID degree, that program has ended but SoA and IDeATe like this class.
What we’re doing differently in a remote class with additional readings and responses/projects.
Syllabus and grading: weekly assignments, three crits, final crit, class participation
Note taking vs. the class blog — will post my notes to the blog after class
Class Theme – “Accessiblity”
Improving the human condition by improving living spaces with tangible interaction design.
Tweaking Nathan Shedroff’s list:
Assist
Enhance
Improve
Qualify
Sense
Class Theme – Quarantine Quitchen
A few months ago, TV personality Alton Brown and his wife, Elizabeth Ingram, started doing a youtube series called “Quarantine Quitchen”. He’s a famous chef, filmmaker, and host of competition cooking shows, she’s an interior designer who does restaurants. Every Tuesday evening (during our class), they make dinner based on whatever they can find in the fridge, freezer, or pantry. And drink some alcohol. I’m lifting their idea of “living in a quarantine” and changing class assignments to reflect being stuck at home, having limited access to physical social events, and social distancing. https://www.youtube.com/playlist?list=PLSL8Njz6ML7Bf5VyF9b0oQ3hiBs_3j-sz
Tangent: explain how PID is different from a sensor + relay
Explain how machine learning (open ended system) is different from PID (a closed loop system)
Questions
What if we had a smart (AI) thermostat? NEST is on the edge of this.
change heating/cooling controls based on history of changes made by people
change temperature related to outside environment: tomorrow is hot, turn the AC on earlier
react to weather changes: storm front rolls through from the north, temp drops almost 10C
modulate temp based on who is in the house: I like it warm, spouse likes it cold
modulate temp based on predicted activities: “they always stay up late on Friday”
error control: “never let the house go below 50F” to prevent pipes from freezing
What if we had a smart door bell? What would it do? This will be an assignment on Thursday.
Short History of Physical Computing and Interaction Design
Physical computing and tangible interaction design are recently created fields but there is a history of how we got here. The key point is the size (scale) of computing hardware.
water powered tools and windmills, literally physical computing
beginning of the PID idea, centrifuges to maintain speed in grain mills
water mills that stop pumping water when the reservoir is full
industrial revolution
early punch-card computing to control looms in 1725
Korsakov proposes using punched cards to store data in 1832
steam engines that can react to malfunctions (more physical computing)
sophisticated PID in factories to run mills and lathes
(skipping mainframes that only crunched numbers)
transistors as second industrial revolution
the first computers that didn’t fill buildings
1976: first use of “human-computer interaction” in a published paper https://www.semanticscholar.org/paper/Evaluating-the-impact-of-office-automation-on-top-Carlisle/7a864fc9cfbb01306cb2a75ceef1ed246727f1f0
1983: The Psychology of Human-Computer Interaction brings the concept to the general computing community https://amzn.to/3jDDVEu
Early arcade games introduced haptic output or feedback: helicopter arcade game that shakes when you are shot (early 1980s)