Setup with no people
People using malleable melodies during the critique
Showing base setup including how wires come out of the bottom of the post
wiring
Judenique using one of the playdough modules during the critique. Shows the underside of the plate with the alligator clips connecting to the aluminum tape nodes feeding into the post.
People made lots of funny sculptures as they interacted with the project
Description
There are 5 yellow posts with some plates and colorful lumps on top. They are all different heights. Each one has a bunch of wires coming out of the bottom that connect to a red box in the middle. There is some strange music playing with a bunch of different sounds. If you change the shape of the colorful lumps (they are mushy) then one of the sounds changes. Each lump changes a different sound in the collection of sounds playing. Because there are 5 you can work with other people to change the different sounds together.
Process
The very first playdough module. Just testing to see if it could work.
First encounter with the corrosion problem. The playdough removed the nice shiny coating from the spring and got dirty.
All of the demo modules I built over the process. You can see the heavy corrosion on the early copper modules as compared to the aluminum one.
Initial “mockup” of the planned installation with a cardboard box acting as the “mother” module into which I would put the computer, arduino, and breadboard.
whiteboard doodles trying to figure out how to deal with the wires coming from the plates and feeding out to the base.
I think one of the main things I learned from this project was the importance of presentation. In my previous project I didn’t focus as much on the “format” of the project, instead I focused more on the “guts,” but in this project I spent a lot of time mocking up various ways of arranging things and I think it really paid off. I also learned the importance of conceptual development not just in the initial ideation stages but also in regards to the overall process. I think something I tend to lose sight of as I work is why I am actually doing what I am doing or what the goal is besides just “finishing” the project, and so when it comes time for the critique I felt like the conceptual underpinning of my project didn’t really match up as well with the experiential aspect. For example my goal was to create a more “flexible” music manipulation software, however I ended up scripting the music people were using, or even having the option of a pause/play feature which I think would have better aligned with that goal. I think if I had set up all five modules earlier and had had people interact with the overall piece at the 90% critique it would have really helped me hone that last 10% to be more aligned with my overall goal.
There were also quite a number of unexpected speed bumps that slowed down the project’s development such as the fact that the playdough proved to be extremely corrosive and would become extremely discolored over time as well as the fact that it is apparently really hard to get the serial port to communicate with p5.js. Despite these setbacks however I think the project was also successful in that users seemed really engaged and drawn to the project, which poses well for further development. I also learned A TON doing this project; I learned how to better plan out and prioritize physical presentation early on, I learned a lot more about how to find and successfully modify code found online to your needs, I learned to have more conceptual discussions throughout the process rather than just at the start/end, and I learned a whole new software.
If I were to continue with this project I’d like to 1) redo the central box to fit in the speakers 2) add some pulsing lights to the center box so it becomes more like a “mother” and the modules part of some sort of superorganism 3) think more about the “flexibility” aspect of the project and maybe modify it so people have some sort of sound selection 4) consider collaborative aspects of the project more and potentialities of different playdough modules interacting with one another.
Code
*** you need to use the program p5.serialcontrol (https://github.com/p5-serial/p5.serialcontrol/releases) in order to use this code. Otherwise arduino code won’t be able to talk to the p5.js code. ***
Arduino Code:
//code for "Malleable Melodies" project by Kirman Hanson as part of 62-362
//code modified from:https://gist.github.com/shfitz/0aabb07daa8ef84904d6e3c6a17381a0
//if you try to get rid of the inbyte feature it won't work
int firstSensor = 0; // first analog sensor
int secondSensor = 0; // second analog sensor
int thirdSensor = 0;
int fourthSensor = 0;
int fifthSensor = 0;
int sixthSensor = 0; // digital sensor, need this for the code to work
int inByte = 0; // incoming serial byte
void setup() {
// start serial port at 9600 bps and wait for port to open:
Serial.begin(9600);
while(!Serial){
;;
}
pinMode(2, INPUT); // digital sensor is on digital pin 2
establishContact(); // send a byte to establish contact until receiver responds
}
void loop() {
// if we get a valid byte, read analog ins:
if (Serial.available() > 0) {
// get incoming byte:
inByte = Serial.read();
// read first analog input:
firstSensor = analogRead(A0);
// read second analog input:
secondSensor = analogRead(A1);
// read third analog input:
thirdSensor = analogRead(A2);
// read fourth analog input:
fourthSensor = analogRead(A3);
//read fifth analog input:
fifthSensor = analogRead(A4);
// read switch value
sixthSensor = digitalRead(2);
// send sensor values:
Serial.print(firstSensor);
Serial.print(",");
Serial.print(secondSensor);
Serial.print(",");
Serial.print(thirdSensor);
Serial.print(",");
Serial.print(fourthSensor);
Serial.print(",");
Serial.print(fifthSensor);
Serial.print(",");
Serial.println(sixthSensor);
}
}
void establishContact() {
while (Serial.available() <= 0) {
Serial.println("0,0,0"); // send an initial string
delay(300);
}
}
p5.js:
//code for "Malleable Melodies" project by Kirman Hanson as part of 62-362
// pitch shift by Anthony T. Marasco [2018] originally from https://codepen.io/lsuddem/pen/RJEYLr
// serial communication with a microcontroller sending multiple values from: https://editor.p5js.org/shfitz/sketches/9dPvGrtEZ
// arduino code [for serial communication] can be found here : https://gist.github.com/shfitz/0aabb07daa8ef84904d6e3c6a17381a0
let serial; // variable for the serial object
let sensors = [255, 255, 255, 255, 255, 255]; // array to hold data from arduino
//sound stuff
let shifter;
let player;
let footPlayer;
let footShifter;
let drumSound;
let drumSound2;
let keySound;
let footsSound;
let bellSound;
let pitchMax = 8.0;
let pitchMin = -15.0;
let bellRateMax = 4.0;
let volMax = 1.0;
let volMin = 0.0;
let footVolMax = 0.7;
let footVolMin = 0.0;
let rateMax = 1.5;
let rateMin = 0.5;
//doughVals
let pD1Map, pD2Map, pD1Round, pD2Round, pD3Map, pD3Round;
//pD4 controls the bells sound volume and rate
let pD4VolMap, pD4VolRound, pD4RateMap, pD4RateRound;
//pD5 controls the footsteps volume and pitch
let pD5PitchMap, pD5PitchRound,pD5VolMap,pD5VolRound;
function preload() {
drumSound = loadSound('drumming.wav');
keySound = createAudio('keys.wav');
footsSound = createAudio('footsteps.mp3');
bellSound = loadSound('bells.wav');
print('preloading');
//storm pitch shift
shifter = new Tone.PitchShift(0).toMaster();
shifter.windowSize = 0.03
player = new Tone.Player('storm.mp3').connect(shifter);
player.loop = true;
//foot pitch shift
footShifter = new Tone.PitchShift(0).toMaster();
footShifter.windowSize = 0.03
footPlayer = new Tone.Player('footsteps.mp3').connect(footShifter);
footPlayer.loop = true;
}
function setup() {
createCanvas(windowWidth, windowHeight);
textAlign(CENTER);
textFont("Helvetica");
fill(0);
//serial stuff
// serial constructor
serial = new p5.SerialPort();
// serial port to use - you'll need to change this
serial.open('/dev/tty.usbmodem14101');
// what to do when we get serial data
serial.on('data', gotData);
// when to do when the serial port opens
serial.on('open', gotOpen);
//sound stuff
drumButton = createButton("Start Drum");
drumButton.position(width / 2 - 50, height / 2);
drumButton.mousePressed(play1);
stormButton = createButton("Start Storm");
stormButton.position(width/2-50, height/2-100)
stormButton.mousePressed(play2)
keysButton = createButton("Start Keys");
keysButton.position(width/2-50, height/2-200)
keysButton.mousePressed(play3)
footsButton = createButton("Start Footsteps");
footsButton.position(width/2-50, height/2-250)
footsButton.mousePressed(play4)
bellButton = createButton("Start Bells");
bellButton.position(width/2-50, height/2+100)
bellButton.mousePressed(play5)
shiftSlider = createSlider(pitchMin, pitchMax, 1, 0.1);
shiftSlider.style("width", "200px");
shiftSlider.position(width / 2 - 100, height / 2 + 70);
footShiftSlider = createSlider(pitchMin, pitchMax, 1, 0.1);
footShiftSlider.style("width", "200px");
footShiftSlider.position(width / 2 - 100, height / 2 + 170);
}
function draw(){
background(255);
noStroke();
//pitch controls
shiftSlider.value(pD1Round);
shifter.pitch = shiftSlider.value();
footShiftSlider.value(pD5PitchRound);
footShifter.pitch = footShiftSlider.value();
//rate controls
bellSound.rate(pD4RateRound);
drumSound.rate(pD3Round);
//volume controls
keySound.volume(pD2Round);
//bellSound.volume(pD4VolRound);
//drumSound.volume(0.7);
footsSound.volume(pD5VolRound);
fill(0);
text('pot1: ' + sensors[0] + ', pot2: ' + sensors[1] + ', pot3: ' + sensors[2] + ', pot4: ' + sensors[3]+', pot5: ' + sensors[4],300,10);
console.log(sensors)
}
function gotData() {
let currentString = serial.readLine(); // store the data in a variable
trim(currentString); // get rid of whitespace
if (!currentString) return; // if there's nothing in there, ignore it
sensors = split(currentString, ',');
//map pD1 to pitch for storm
pD1Map = map(float(sensors[0]), 200, 800, pitchMin, pitchMax);
pD1Round = round(constrain(pD1Map, pitchMin, pitchMax),1);
//map pD2 to volume for keys
pD2Map = map(float(sensors[1]), 95, 300, volMin, volMax);
pD2Round= round(constrain(pD2Map, volMin, volMax), 1);
//map pD3 to rate for drums
pD3Map = map(float(sensors[2]), 100, 300, rateMin, rateMax);
pD3Round= round(constrain(pD3Map, rateMin, rateMax), 1);
//map pD4 to volume and rate for bells
pD4VolMap = map(float(sensors[3]), 100, 500, volMin, volMax);
pD4VolRound= round(constrain(pD4VolMap, volMin, volMax), 1);
pD4RateMap = map(float(sensors[3]), 100, 500, rateMin, bellRateMax);
pD4RateRound= round(constrain(pD4RateMap, rateMin, bellRateMax), 1);
//map pD5 to volume and pitch for foots
pD5VolMap = map(float(sensors[4]), 100, 300, footVolMin, footVolMax);
pD5VolRound= round(constrain(pD5VolMap, footVolMin, footVolMax), 1);
pD5PitchMap = map(float(sensors[4], 100, 300, pitchMin, pitchMax),1);
pD5PitchRound = round(constrain(pD5PitchMap, pitchMin, pitchMax), 1);
//console.log("pD1Round:"+pD1Round);
serial.write('A');
}
function gotOpen() {
print("Serial Port is Open");
serial.clear(); // clears the buffer of any outstanding data
serial.write('A'); // send a byte to the Arduino
}
function play1() { // set up as drum & rate button
drumSound.loop();
}
function play2(){ //set up as storm & pitch player
player.start();
}
function play3(){//set up as keys/volume player
keySound.loop();
}
function play4(){//set up as footsteps/volume player
footsSound.loop();
footPlayer.start();
}
function play5(){//set up as footsteps/volume player
bellSound.loop();
}
p5.js index code:
<!DOCTYPE html>
<html lang="en">
<head>
<script src="path/to/p5.sound.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/p5.js/1.4.0/p5.js"></script>
<script language="javascript" type="text/javascript" src="https://cdn.jsdelivr.net/npm/p5.serialserver@0.0.28/lib/p5.serialport.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/p5.js/1.4.0/addons/p5.sound.min.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/tone/13.2.0/Tone.min.js"></script>
<link rel="stylesheet" type="text/css" href="style.css">
<meta charset="utf-8" />
</head>
<body>
<script src = "comboSketch.js"></script>
</body>
</html>