Project 01 : Phipps Conservatory Lights


The Winter Flower Show and Light Garden is Phipps Conservatory’s largest event of the year and draws over 120,000 visitors throughout its six week run. Phipps has recently installed a dynamic lighting installation in the Sunken Garden room and is seeking proposals for an interactive or reactive extension of the lighting system.
This project requires that groups design and build a fully-functioning prototype for the room and demo that prototype on site for the organization’s director. The director will select the most viable prototype to be further developed and temporarily installed in the Sunken Garden for the duration of the six week festival.


Jordyn Melino, the Exhibit Coordinator and lead designer of the Light Garden, has provided us with two of the drawings they’ll be using to build the room. The first is the planting plan, which depicts all of the plants and locations for the room. Color is a major part of the show and the plants and decor will focus on reds and golds.

The second drawing depicts prop inspirations. The room’s theme will be one of a holiday feast, with props and plants that represent a holiday table setting.

Phipps Lighting Test from ULRStudio on Vimeo.


Because of the high visitor volume throughout the show, the Sunken Garden is prone to long lines. The goal of this project is two-fold:

  1. Enhance and augment the whimsical decorations of the room.
  2. Entertain visitors as they queue up to pass through the next room.


  1. Groups of 4 or 5 will design and develop a functioning prototype to present on October 15th.
  2. Projects interact with the lights using our class’s custom API
  3. No screen-based projects
  4. At least one sensor input (this can be a microphone, a board of potentiometers, buttons, etc)

Selection Rubric

  1. Robustness: will a final version of this project be able to stand up to 120,000 people in six weeks?
  2. Integration: how well does this project mesh with the existing decor of the room (color, whimsy, fun, etc)
  3. Timing: is this something that won’t hinder the already slow-moving lines? Can visitors enjoy this without it clogging up the space?


  1. Thursday, September 28: Class tour of Phipps, Q&A with Jordyn Melino
  2. Tuesday, October 3: Design proposal blog post due, review in class
  3. Tuesday, October 3: Introduce lighting API and simulator
  4. Thursday, October 5 – 12: In-class development
  5. Thursday, October 12: In-class demo with lighting simulation – prototype must be functional!
  6. Sunday, October 15: Evening demos at Phipps

Mini Design Exercise 02

Kinetic Data Sculpture


Using the technical skills we’ve developed over the past three weeks, design and prototype a kinetic, sculptural visualization of a remote data stream. Legibility is NOT a requirement for this project. The goal here is to play with form and consider alternative implementations of the physical->digital->physical communication pipeline.


  1. Final project is due at the end of class.
  2. Using the simple materials at hand, design a kinetic sculpture that changes its motion based on a remote, external, physical input.
  3. System must use two computers that speak to each other over a network
  4. One computer must pull in sensor data (either through arduino or the computer’s microphone)
  5. The other computer must control the motion of your physical design.
  6. Take a short video of the sculpture and post it to the blog.

Network Communication

The Network

The OSI model.

The layers of our common networking system are dense, complicated and not worth digging into at this time.


IP Address

Each connected computer or microcontroller in a given network is considered a “device” or “host“. Each device in a given network will have an IP address. The IP address can be assigned by the device itself (setting a Static IP) or the address can be given to the device by a router using a process called DHCP. DHCP is most common in large wifi networks and is probably what your laptop is using now. We call this “dynamic.”

When communicating with an application on a different computer, you’ll need to know that computer’s IP address. If both computers are on the same network (same router or same large institutional network like CMU), connecting shouldn’t be a problem.

Networks almost always regulate how their devices access the internet. This means that directly communicating with a device outside of your immediate network can be tricky.

For most applications in this class, you’ll be working with devices on the same router. To make sure you have a clear communication path between devices, check the IP address on each. You can check this by typing ifconfig in the command line. Look for an inet address that is not (which is your ‘localhost’ address, used for talking to other applications on the same device). The inet address may have an adddress such as or Communication shouldn’t be a problem if both devices are on the same subnet, which means the first 4 numbers of the IP address. Keep in mind, this only holds true if both devices are on the same network; so if Kim is in Indiana and his router gives him an IP of and Pat is in Oregon and her router gives her an IP of, that doesn’t mean they can communicate, since their routers are on different networks (assuming no VPN, etc).


If you imagine that and IP address is like a device’s street address, the Port is like a given application’s apartment number within the device. When setting up a networking path on two applications, the listening application will need to declare which port it is listening to. This is usually a 4 or 5 digit number 12345 or 5555, etc. The sending application will also need to declare, along with the IP address, which port in the destination device its packets are intended to reach.

Port numbers less than 1000 are generally reserved and should not be used in your application. You should be aware of some specific ports, like 80 (which is the default port for HTTP) and 22 (which is reserved for SSH).



User Datagram Protocol is a “transport level” protocol we often use for streaming continuous data. UDP is low latency and it does not require that the sender of UDP receive confirmation of receipt from the receiver. This means that UDP messages can get lost and nobody will know. We use UDP for streaming continuous information (people tracking, continuous sensor data, etc) because we want to get that information as soon as possible and we don’t care if packets are lost along the way because another packet will be coming right behind it.

UDP Drop Example


Open Sound Control is an “application level” protocol that sits on top of UDP and allows users to send formatted, human-readable messages. An example of an OSC message may be /sensors/A/value 54.2145 or /heartbeat "I'm alive" 20170206 0.15. We often use OSC to stream sensor data between applications, as it is easier to read and route than plain UDP. OSC does have some overhead, so if you need to send a ton of data and bandwidth is an issue, you may want to develop a protocol based on straight UDP.

Test It!

  1. In Processing, install the oscP5 package.
  2. Open the oscP5sendReceive example (file>examples>contributed libraries>oscP5>oscP5sendReceived)
  3. Find your IP address (on macOs, hold the option key while clicking the wifi icon)
  4. Give your IP address to a friend, get an IP address from a friend.
  5. on line 26 of the example, myRemoteLocation = new NetAddress("",12000); change to your friend’s IP address.
  6. Run the example.
  7. Modify the messages and protocol that you send each other.

Sample Code From Class


Transmission Control Protocol is the “transport level” protocol on which HTTP runs. TCP is slower than UDP because it requires a confirmation of receipt for each message. We use TCP for sending information that MUST reach its destination. For example if we want to send updates from our FitBit to a server every 10 minutes, we would use a protocol based on TCP because if one of those FitBit update packets didn’t arrive, our system might fail. We usually do not use TCP for streaming continuous data.

At Home OSC Exercise


Build a closed system with a partner in which two computers speak to each other using OSC.

– One computer must read a sensor.
– Send the sensor data to the other computer.
– The other computer must use that data to control a physical output.
– Post a video of your system to the blog.

Transistors + Motors

Transistors & Actuators


Since you shouldn’t (and usually can’t) power high-draw devices using the Arduino, you should power them via an external power supply. But what if you want to control when they receive power? You can do that with a Transistor.

Transistors are essentially switches, except they are switched open and closed with electricity instead of a mechanical action. They exist in two main forms (at least with what we’ll be using): BJTs (Bipolar Junction Transistor) and MOSFETs (Metal-Oxide Semiconductor Field-Effect Transistor).

This example shows a high-power LED being powered by a 5V power source, but being controlled by a Transistor hooked up to a microcontroller. When the control voltage goes HIGH, the electricity is allowed to pass through. When it goes LOW, no electricity can pass through the transistor, so the LED can’t turn on.

There are two types of BJTs — NPNs and PNPs.

In diagrams, the arrows for both are on the Emitters. On an NPN, the arrow is NOT POINTING IN. On a PNP, the arrow DOES POINT IN.

On an NPN transistor, the Emitter is connected to Ground, Collector is connected to the Current source you want to pass through, and the Base is connected to the Current source you want to use to control the Transistor. In PNP transistors, the Emitter and Collector are switched.

MOSFETs are very similar, except their pins are named Source, Drain, and Gate. Additionally, the two types are N-Channel and P-Channel.

In diagrams, the arrows are always on the Gate. On N-Channel MOSFETS, the arrow points IN, and on P-Channels it points OUT.

In an N-Channel MOSFET, the Source is connected to ground, the Drain is connected to your high-powered circuit, and the Gate is connected to the control to switch the MOSFET on or off. In P-Channel MOSFETS, the Source and Drain are switched.

DC Motors

DC Motors are a very simple type of motor that turns at varying speeds depending on how much voltage is supplied.

They are exteremely simple to use. Simply connect one wire to power, and the other wire to ground to make the motor spin one way. Flip the wires around to make it spin the other way. You can determine what Voltage to use by looking at the motor’s datsheet.

These motors, for example, have a suggested Voltage of 4.5V DC, and will pull 250mA of current at max load.

Exercise 2.1: Using a Transistor to Control a DC Motor

  1. Connect your motor to a Power Supply, and give it 4.5V to see it spin. Flip the wires around the other way to see it spin backwards
  2. Design (draw or make in Fritzing) and build a circuit using an N-Channel MOSFET to power that motor via a Power Supply, and control it via an Arduino (you should use the Blink example as a starting point)


Servos are a little more complicated than regular DC motors. While generally not as strong, you can control the exact speed at which a servo turns, as well as the direction and position it turns to.

To do this, Servos have three wires: Ground, Power, and Control. The colors vary on different servos, but for the most part, red is usually power, black/brown is usually ground, and white/yellow is usually control.

Servos have some limitations. Most servo motors cannot rotate continuously; cheaper servos can have as little as 90° of rotation. Some servos, however, are known as continuous rotation, and can rotate in full circle indefinitely.

Exact control of a hobby servo is accomplished by sending a series of pulses to the device over the control line. We usually don’t generate our own pulses (they’re difficult to keep track of), and instead rely on existing libraries that have all of the potential commands already written.

Exercise 2.2: Controlling a Servo with a Potentiometer

  1. Connect a servo to your Arduino like so:

  1. Open the example code, File > Examples > Servo > Sweep, and upload it to your Arduino
  2. Attach a Potentiometer to your system, and use it to control the exact positioning of the Servo (make use of the map() command)


Joshua Noble’s Circuit Diagram Symbols – Programming Interactivity
Tom Igoe’s Understanding Electricity
Sparkfun’s Voltage, Current, Resistance, and Ohm’s Law, How to Read a Schematic, Motors and Selecting the Right One, and Hobby Servo Tutorial
ITP’s Electricity: the Basics
cmuphyscomp’s physcomp-examples
Oscar Liang’s How to Use MOSFET — Beginner’s Tutorial

Phys-Digi-Phys In Class Exercise


Pulling in data, manipulating that data with a series of algorithms or interactions, and then translating that modified data into a different medium is at the heart of what we are going after in this class. This mini-exercise will be our first attempt at experimenting with this type of data-flow.
For this exercise, you will need to build and program a simple system that pulls in information about the physical world from a sensor using Arduino, analyzes or modifies that information in Processing, and outputs that information in a physical way, either through light, motion, or sound.

Groups of Two.


  1. Include one or more physical sensors.
  2. Modify or analyze the data coming in from that sensor.
  3. Manipulate the data from the sensor interactively (via keyboard, mouse, audio, etc).
  4. Output that data in a new medium (LED, piezo buzzer, motor, servo, etc.).
  5. Document your experiment and post a video to the blog. Include both partners and a simple hand drawing of how it works.

Intro to Arduino

Each Group Collect

  • Digital Multimeter
  • Variable Voltage Power Supply
  • Bana Plug Leads
  • Resistors
  • DC Motor

What is Electricity?

A general understanding of electricity and the rules that determine how it flows and how it can be controlled are crucial to quickly prototyping your designs. These resources cover the fundamentals of working with electricity.

Water analogy from Sparkfun

Voltage = Pressure in system


Current (Amperage) = Amount of Flow



Resistance = Resistance to Flow

V = I * R (voltage = current * resistance)

I = V / R (current = voltage / resistance)

Solve for LED / Resistor Combo

We know the ideal forward voltage (Vf) and current (I) of the LED.

Target Vf = 3.3v

Target I = 20mA (0.02A)

V = I * R

R = (Vs – Vf) / I

Exercise 1: Learn the Digital Multimeter (DMM)

Sparkfun: How to Use a Multimeter

  1. Set up a simple circuit on a breadboard with an LED, a resistor, and the variable voltage power supply:
    1. Anode (+) [because Node.js is positive/great]
    2. Cathode (-) [because cats are negative/terrible]
  2. Set the power supply to 5V
  3. Use the the DMM to measure voltage at different points of the circuit
  4. Notice the amount of current being used by the power supply
  5. Insert the DMM into the circuit and measure the amount of current (see if it matches what the power supply says)
  6. What happens if you change the resistor?
  7. Add a switch or button
  8. Draw the [circuit diagram]( on the table


What is Arduino?

From: Instructables

A simple microcontroller and software combo developed to help designers, artists and non-engineers rapidly prototype with electronics.

Some things to remember:

  1. Analog Pins are for reading only (0v – 5v)
  2. analogRead(pinNumber) returns an integer value between 0 – 1023
  3. Digital Pins can be INPUT or OUTPUT
    1. digital pins can supply a low-current (100mA) 5v supply in output mode:  digitalWrite(pin, HIGH/LOW);
    2. digital pins can read the state of that pin, as either HIGH or LOW (5v or 0v): digitalRead(pin);
  4. You can run the arduino WITHOUT the USB cable attached (only a 5v-24v barrel jack)
    1. You can also power from the Vin and GND pins.


Exercise 2: Set up Arduino & Blink

  1. Download the Arduino IDE
  2. Follow the Getting Started guide for the Arduino UNO (word for word!)
  3. Run the Blink! example sketch
    1. Open Examples > Basics > Blink
    2. Plug your Arduino into the computer
    3. Choose your board: Tools>Board
    4. Choose the correct port: Tools>Port
    5. Click the check icon (verify)
    6. If no errors, upload to the arduino

Exercise 3: Reading Data & Graphing it

  1. Upload the Standard Firmata to your Arduino (check out File>Examples>Firmata>StandardFirmata)
  2. Make sure you have the Arduino(Firmata) library (you can download this from the Sketch>Import Library>Add Library… menu)
  3. In Processing and copy/paste this code:

  1. Find a sensor (Potentiometers are a good one)
  2. Attach it to the Analog pins of your Arduino (You may need to find a diagram/datasheet/instructions on the internet)
  3. Run the Processing sketch!


Project 0: Mini Design Exercise

Client: Google
Location: Bakery Square 2.0 Bridge, 6425 Penn Ave, Pittsburgh, PA
Due: Thursday 8/31


Google currently leases several floors in two adjacent buildings on either side of Penn Avenue. These two buildings are connected by an enclosed pedestrian bridge on the 5th floor. Google is requesting proposals for a reactive, experiential installation to be integrated into the bridge.


  • Create a joyous experience for those passing through.
  • Integrate Google technologies or data
  • Allow for “passive” interaction
  • Give employees a break from work if they want it.
  • Don’t be obtrusive for those in a hurry.
  • The installation must be experienced from both inside the bridge and outside on the street below.

Site Images:





  • Low fidelity sketch of concept
  • Interaction diagram (show us how people/data/environment influence the installation, and how the installation influences people)
  • 1 minute pitch
    • pretend the class is the client, pitch this concept to us in VERY BRIEF terms.

Welcome to RSMA Fall 2017

Welcome to the Fall 2017 installment of Reactive Spaces and Media Architecture. The goal of this course is to develop a visual and conceptual vocabulary and authority around placing dynamic media in space. We’ll accomplish this task through a series of group and individual projects, precedent research, readings and discussions, and group critiques of our work and the work of others.

Along they way, be prepared to learn new skills such as working with micro processors, integrating with commercial lighting systems, accessing APIs, basic electronics, and programming languages like Processing or node.js.

You won’t need any programming or electronics experience to succeed in the class. However, if you’ve never written code or blinked an LED, you may have to put in some extra time learning the new skills on your own.


What To Expect?

Examples :
Simon Heijdens, Shade
Ethan Rose, Elements
Refik Anadol, Virtual Depictions
Ned Kahn, Articulated Cloud
Sosolimited, Criss-Cross Signal Spire
Studio Roosegaarde, Windlicht

What We’re Doing

Traditional Method (Triggers + Timelines)

Our Method (Realtime Data Processing)

What Have Students Done Before?

Tangible Lightwall from ULRStudio on Vimeo.

Ghost Pier from Heidi Yang on Vimeo.

Mutable Landscape from Leah Wulfman on Vimeo.

Audio Window from Nickie Cheung on Vimeo.