Month: December 2018

My final project uses the webcam to generate simple animations that will follow motion/gestures of the user through a red color marker. For the best experience, make sure to only have one red object in front of the camera at a time.

Instructions:

1. grab something red (any relatively small red object would work best, or just pull up a red image on your phone)

2. hold it within range of the webcam, with the red facing the camera

3. wave/draw/play

   *to see the animations alone, you can turn the camera image on/off by pressing camera button

   press on shape 1, shape 2, and shape 3 to see different shapes

   works best in a brightly lit room, so the computer can recognize the red object

sketch

// Sophie Chen
// sophiec@andrew.cmu.edu
// 15-104 Final Project
// Dec 2018

// This program uses the webcam and generates a simple animation that will
// follow a red color marker. For best user experience, make sure to only have
// one red object in front of the camera at a time.

// declare variables
var myCaptureDevice;
var xarray = []; // array to store the value of x positions
var yarray = []; // array to store the value of y positions
var limitSize = 50; // length limit of x and y arrays
var cameraOn = 1; // camera switch - default on
var shapeOne = 1; // default starts with shape one
var shapeTwo = -1; // shape two starts false
var shapeThree = -1; // shape three starts false
var turtle; // define turtle graphics variable


// setup loads the webcam input and initializes turtle graphics
function setup() {
    createCanvas(600, 430);
    myCaptureDevice = createCapture(VIDEO);
    myCaptureDevice.size(600, 430);
    myCaptureDevice.hide();
    turtle = makeTurtle(0, 0);
    turtle.penDown();
    turtle.setColor(255);
}

// this function tests for color as a condition
function isColor(c) {
	return (c instanceof Array);
}

// this function draws the animations on top of the camera image
function draw() {
    myCaptureDevice.loadPixels();
    // if cameraOn is true, load camera output
    // if cameraOn is false, camera output is not visible, load black background
    if (cameraOn === 1){
        image(myCaptureDevice, 0, 0);
    } else {
        background(0);
    }

    //call functions that draw the buttons
    drawCamButton();
    drawShapeButton();
    drawShapeTwoButton();
    drawShapeThreeButton();

    //declare variables used to calculate centerpoint of red object/marker
    var xMin = 0; // x value minimum 
    var xMax = 600; // x value maximum
    var yMin = 0; // y value minimum
    var yMax = 430; // y value maximum

    // for loop that draws the shape animations
    for (var a = 0; a < xarray.length; a++){ 
        // declare color and size variables based on forloop
        var size = (50 / xarray.length) * a;
        var r = map(a, 0, xarray.length, 0, 255);
        var g = map(a, 0, xarray.length, 0, 255);
        var b = map(a, 0, xarray.length, 0, 255);

        // Shape 1: filled ellipse
        if (shapeOne === 1){
            shapeTwo = -1;
            shapeThree = -1;
            noStroke();
            fill(r, g, 255);
            ellipse(xarray[a], yarray[a], size, size);
        }


        // Shape 2: outlined ellipse
        if (shapeTwo === 1) {
            shapeOne = -1;
            shapeThree = -1;
            noFill();
            stroke(r, g, b);
            strokeWeight(1);
            ellipse(xarray[a], yarray[a], size, size);
        }

        
        // Shape 3: turtle graphics
        if (shapeThree === 1) {
            shapeOne = -1;
            shapeTwo = -1;
            turtle.setColor(color(205, 255, 10));
            turtle.goto(xarray[a], yarray[a]);
            turtle.forward(25);
            turtle.right(90);  
        }
    }

    // get the color value of every 5 pixels of webcam output
    for (var i = 0; i < width; i += 5){
        for (var j = 0; j < height; j+= 5) {
            var currentColor = myCaptureDevice.get(i, j);
            // targetColor: color(255, 0, 0);

            // calculate the difference between current color and target color
            var dr = red(currentColor) - 255;
            var dg = green(currentColor) - 0;
            var db =  blue(currentColor) - 0;
            
            // if current color is close enough to target color (~120), calculate
            // center point of the red area
            if (isColor(currentColor)){
               var dist = sqrt(sq(dr) + sq(dg) + sq(db));
               if (dist < 120) {
                    // find center point of red marker
                    if (i > xMin){ 
                        xMin = i;
                    }
                    if (i < xMax){
                        xMax = i;
                    }

                    if (j > yMin){
                        yMin = j;
                    }
                    if (j < yMax){
                        yMax = j;
                    } 
                }   
    		}
    	}
    }

    // push the newly discovered x, y into the array 
    xarray.push((xMin + xMax) / 2);
    yarray.push((yMin + yMax) / 2); 
    
    // if array is full, pop something out from the beginning
    while (xarray.length > limitSize) {
        xarray.shift();
        yarray.shift();
    }
}

// functions to trigger responses of buttons pressed
function mouseClicked(){
    // if camera button is pressed, toggle on/off
    if (mouseX > 10 & mouseX < 60 && mouseY > 10 && mouseY < 30){
        cameraOn = -cameraOn;
    }
    // if shape 1 button is pressed, show shape 1, disable shape 2 & 3
    if (mouseX > 10 && mouseX < 60 && mouseY > 20 && mouseY < 60){
        shapeOne = 1;
        shapeTwo = -1;
        shapeThree = -1;
    }
    // if shape 2 button is pressed, show shape 2, disable shape 1 & 3
    if (mouseX > 10 && mouseX < 60 && mouseY > 60 && mouseY < 90){
        shapeTwo = 1;
        shapeOne = -1;
        shapeThree = -1;
    }
    // if shape 3 button is pressed, show shape 3, disable shape 1 & 2
    if (mouseX > 10 && mouseX < 60 && mouseY > 90 && mouseY < 120){
        shapeThree = 1;
        shapeTwo = -1;
        shapeOne = -1;
    }

}

// camera button
function drawCamButton(){
    fill(255);
    stroke(0);
    rect(10, 10, 50, 20);
    noStroke();
    fill(0);
    text('camera', 15, 23);
}

// shape 1 button
function drawShapeButton(){
    fill(255);
    stroke(0);
    rect(10, 40, 50, 20);
    noStroke();
    fill(0);
    text('shape 1', 15, 54);
}

// shape 2 button
function drawShapeTwoButton(){
    fill(255);
    stroke(0);
    rect(10, 70, 50, 20);
    noStroke();
    fill(0);
    text('shape 2', 15, 85);
}

// shape 3 button
function drawShapeThreeButton(){
    fill(255);
    stroke(0);
    rect(10, 100, 50, 20);
    noStroke();
    fill(0);
    text('shape 3', 15, 116);
}

//////////////////////////////////////////////////////////////////////////////
function turtleLeft(d){this.angle-=d;}function turtleRight(d){this.angle+=d;}
function turtleForward(p){var rad=radians(this.angle);var newx=this.x+cos(rad)*p;
var newy=this.y+sin(rad)*p;this.goto(newx,newy);}function turtleBack(p){
this.forward(-p);}function turtlePenDown(){this.penIsDown=true;}
function turtlePenUp(){this.penIsDown = false;}function turtleGoTo(x,y){
if(this.penIsDown){stroke(this.color);strokeWeight(this.weight);
line(this.x,this.y,x,y);}this.x = x;this.y = y;}function turtleDistTo(x,y){
return sqrt(sq(this.x-x)+sq(this.y-y));}function turtleAngleTo(x,y){
var absAngle=degrees(atan2(y-this.y,x-this.x));
var angle=((absAngle-this.angle)+360)%360.0;return angle;}
function turtleTurnToward(x,y,d){var angle = this.angleTo(x,y);if(angle< 180){
this.angle+=d;}else{this.angle-=d;}}function turtleSetColor(c){this.color=c;}
function turtleSetWeight(w){this.weight=w;}function turtleFace(angle){
this.angle = angle;}function makeTurtle(tx,ty){var turtle={x:tx,y:ty,
angle:0.0,penIsDown:true,color:color(128),weight:1,left:turtleLeft,
right:turtleRight,forward:turtleForward, back:turtleBack,penDown:turtlePenDown,
penUp:turtlePenUp,goto:turtleGoTo, angleto:turtleAngleTo,
turnToward:turtleTurnToward,distanceTo:turtleDistTo, angleTo:turtleAngleTo,
setColor:turtleSetColor, setWeight:turtleSetWeight,face:turtleFace};
return turtle;}

I really enjoyed the camera interaction aspect of the text rain assignment, which is why I wanted to work more with live cam for this project and create something where the user has even more direct control. Overall this project was a lot more challenging than I expected, but that also made it more rewarding when I finally got it to work. I’m glad I went with color recognition for the marker because it allows for precise control and is more forgiving towards the user in terms of what environment they should be in when using this program. The most time-consuming and unexpected challenge was the lagging and freezing that comes with working with so many pixels, so trying to figure out what was causing the freezing took a lot longer than changing the code to fix it. Since the animations are on the simple side, I decided to include 3 different options. Ideally the animations would’ve been more complex, that’s something I hope to keep working on in the future. Below are screenshots from me using the program to give an idea of how it would look like when it’s working.

Term Project

// Joanne Lee
// Section C
// joannele@andrew.cmu.edu
// Term Project

// Visual Quadrant Map:
// Q1  Q2  Q3
// Q4  Q5  Q6
// Q7  Q8  Q9

// logistics
var game = "inactive"
var fCount = 150;
var playBoardW = 270;

// playing board variables
var boardWeight = 10;
var playBoxW = 270;
var effectCounter = 6;

// white, orange, gray box variables
var boxW = playBoardW / 4;
var boxX = [165.625, 250, 334.375];
var boxY = [240.625, 325, 409.375];
var grayBoxes = [];
var newOrange = false;
var whiteX = 0;
var whiteY = 2;
var orangeX = 2;
var orangeY = 0;

// score keeper
var score = 0;
var bestScore = 0;

// sound variables
var coin;
var boing;

function preload() {
    coin = loadSound("https://courses.ideate.cmu.edu/15-104/f2018/wp-content/uploads/2018/12/coin.wav");
    coin.setVolume(0.5);
    boing = loadSound("https://courses.ideate.cmu.edu/15-104/f2018/wp-content/uploads/2018/12/boing.wav");
    boing.setVolume(0.5);
}

function setup() {
    createCanvas(500,650);
    frameRate(100);
}

function draw() {
    background(255,211,222);

    // "growth" effect seen when white + gray boxes collide
    if (effectCounter < 2) { // staggered growth effect
        playBoxW = 280;
    }
    if (effectCounter > 2 & effectCounter < 4) { // staggered growth effect
        playBoxW = 290;
    }
    if (effectCounter > 4 & effectCounter < 6) { // staggered growth effect
        playBoxW = 280;
    }
    if (effectCounter > 6) { // staggered growth effect
        playBoxW = 270;
    }
    effectCounter += 1;

    drawPlayingBoard();
    drawPlayerBox();
    drawGoalBox();
    displayScore();

    if (game == "inactive") {
        displayStarterText();
    }
    if (game == "active") {
        drawGrayBox();
        // control speed that new boxes are generated
        if (frameCount % fCount == 0 & frameCount != 0) {
            generateGrayBox();
        }
        updateGrayBox();
        checkCollision();
    }
}

function displayScore() {
    fill(255,80); // display current score
    textSize(200);
    textStyle(BOLD);
    textFont('Helvetica');
    if (score < 10) {
        text(score, width - 120, height - 10);
    }
    else if (score < 100) { // shift over one spot for 2 digit score
        text(score, width - 225, height - 10);
    }
    else if (score < 999) {
        text(score, width - 330, height - 10 );
    }
    if (score >= bestScore) { // record the best score
        bestScore = score
    }

    fill(255); // display the best score
    textSize(20);
    textStyle(BOLD);
    textFont('Helvetica');
    text("High Score  " + bestScore, 110, height-160);
}

function displayStarterText() {
    fill(40,99);
    textSize(20);
    textStyle(BOLD);
    textFont('Helvetica');
    text("Press enter to begin", 155, height/2);
}

function drawPlayingBoard() { // light pink w/ thick white border
    fill(255,211,222);
    stroke(255); 
    strokeWeight(boardWeight);
    rectMode(CENTER);
    rect(width / 2, height / 2, playBoxW, playBoxW);
}

function drawPlayerBox() { // white box, starts in Q7
    strokeWeight(0);
    fill(255);
    rect(boxX[whiteX], boxY[whiteY], boxW, boxW);
}

function drawGoalBox() { // orange box, starts in Q3
    fill(255,170,107);
    rect(boxX[orangeX], boxY[orangeY], boxW*0.7, boxW*0.7);
}

function randomOrangeXY() { // generate random position for orange box
    var randX = int(random(0,3));
    var randY = int(random(0,3));

    // make sure orange x & y are in same line or column as previous position
    while (randX == whiteX || randY == whiteY) {
        randX = int(random(0,3));
        randY = int(random(0,3));
    }
    orangeX = randX;
    orangeY = randY;
}

function checkBoard() {
    // if white box position = orange box, you need a new orange box
    if (whiteX == orangeX & whiteY == orangeY) {
        coin.play();
        newOrange = true; // indicate need for new orange box
        score += 1; // increment score

        // increase speed at which new boxes spawn, cap at 90 frame count
        if (score % 10 == 0 & score != 0 && fCount > 90) {
            fCount -= 20;
        }   
    }

    // if board needs new orange box, randomly generate
    if (newOrange == true) {
        randomOrangeXY();
        newOrange = false;
    }
}

function generateGrayBox() { // will create a box with random attributes
    if (int(random(0,2)) == 0) { // create a box going left or right
        if (int(random(0,2)) == 0) { // create box going right
            createBox(-23.625, boxY[int(random(0,3))], 1, 0);
        }
        else { // create box going left
            createBox(523.625, boxY[int(random(0,3))], -1, 0);
        }
    }
    else { // create a box going up or down
        if (int(random(0,2)) == 0) { // create box going down
            createBox(boxX[int(random(0,3))], -23.625, 0, 1);
        }
        else { // create box going up
            createBox(boxX[int(random(0,3))], 673.625, 0, -1);
        }
    }
}

function createBox(tx, ty, dx, dy) { // store as an object in the array
    var grayBox =   { x: tx, y: ty,
                      dx: dx, dy: dy,
                      speed: 2.5, state: 'active'
                    }
    grayBoxes.push(grayBox);
}

function updateGrayBox() { // moves the box along
    for (var a = 0; a < grayBoxes.length; a ++) {
        grayBoxes[a].x += grayBoxes[a].dx * grayBoxes[a].speed;
        grayBoxes[a].y += grayBoxes[a].dy * grayBoxes[a].speed;
    }
}

function drawGrayBox() {
    for (var b = 0; b < grayBoxes.length; b ++) {
        fill(107,105,107);
        rect(grayBoxes[b].x, grayBoxes[b].y, boxW*0.7, boxW*0.7);
    }
}

function checkGrayBox() {
    for (var c = 0; c < grayBoxes.length; c ++) {
        // delete box from array once it is off the screen
        if (grayBoxes[c].x < -24 || (grayBoxes[c].x > 524) || 
            (grayBoxes[c].y < -24) || (grayBoxes[c].y > 674)) {
            count += 1;
            grayBoxes.splice(c,1);
        }
    }
}

function checkCollision() {
    for (var d = 0; d < grayBoxes.length; d ++) {
        if ((grayBoxes[d].x + boxW * 0.35) > (boxX[whiteX] - boxW * 0.5) &
            (grayBoxes[d].x - boxW * 0.35) < (boxX[whiteX] + boxW * 0.5) &&
            (grayBoxes[d].y - boxW * 0.35) < (boxY[whiteY] + boxW * 0.5) &&
            (grayBoxes[d].y + boxW * 0.35) > (boxY[whiteY] - boxW * 0.5)) {
            gameReset();
            break;
        }
    }
}

function gameReset() {
    boing.play();
    score = 0;
    grayBoxes = [];
    fCount = 150;
    effectCounter = 0;
    game = "inactive";
}

function keyPressed() {
    if (game == "active") {
        if (keyCode == UP_ARROW & whiteY > 0) {
            whiteY -= 1;
        }
        if (keyCode == DOWN_ARROW & whiteY < 2) {
            whiteY += 1;
        }
        if (keyCode == LEFT_ARROW & whiteX > 0) {
            whiteX -= 1;
        }
        if (keyCode == RIGHT_ARROW & whiteX < 2) {
            whiteX += 1;
        }
    }
    if (game == "inactive" & keyCode == ENTER) {
        game = "active"
        generateGrayBox();
    }

    // CHECK BOARD: to see if white box "ate" orange box
    checkBoard();
}

The objective of the game is quite simple: collect the orange box by using your arrow keys to move the white box while avoiding the incoming gray boxes!

As the game progresses, you’ll find that the gray boxes will appear quicker, creating more obstacles to dodge. The game was based off of a game I saw a while ago, but was unable to find again on the internet — so the idea is not original, however, the interface is!

Although seemingly simple, there were a lot of technical obstacles I had to overcome with the main one being having to make sure that too many gray boxes did not appear at once. I ran into some bugs along the way, but I believe that my current code is clean and efficient!

Something I also tried to focus on was simple, but meaningful UI. Attention to detail was put in to small things such as a visual (and audio) cue when you mess up or the placement of the score. Hopefully this is a soothing yet addictive game that will get people through finals week! It was fun to create something of my own.