Author: galsanea@andrew.cmu.edu

Project Interface

I created a smart watch interface for someone with anxiety and panic disorder. It reads someone’s pulse and breath rate and can detect when someone is having a panic attack. I used real-time biometric data from a pulse sensor and a wind sensor and hooked it up to an ardiuno, then I used serial communication to send the data and trigger & display different calming graphics using p5.js. I will go through the project interface and design first, then I will show you how I went about doing it.

Regular Interface

Interface during a panic attack

Depending on the kind of panic attack you are having, there are 3 modes/techniques. The good thing is that any of these techniques would work during a panic attack. I weighted certain techniques to be triggered based on someone’s sensor data. But for the purpose of the scope of this project, and because this requires more knowledge, time and research, I used keyboard keys to to demonstrate the different interfaces.

1. Deep Breathing Technique (for when you’re hyperventilating)
2. Counting Technique (for when your heart rate cannot be controlled)
3. Distraction Technique (for when you need to be distracted from your overwhelming thoughts)

These techniques and methods were based on my research on medical articles here, here, and here.

Deep Breathing

Counting

“Recently I found that trying to focus on something else such as counting numbers… works, because it requires concentration.”

Melanie Luxenberg

Distraction

Proof of Concept

How to Implement it

I used the step by step guide provided by NYU’s physical computing department here to learn how to use ardiuno and p5.js together. If you follow that guide, you can know how to download and run the project. You will need to download the P5.js complete library, the P5.serialserver, and the ardiuno software.

You can download all the files you need to run the project here:

galsanea_FinalProject_allFiles

WordPress Version

If you just want to see the p5.js interaction, I modified the project to run on WordPress using mouse X location to simulate breathing pattern, and mouse Y location to simulate pulse patterns.

Use the following to replace the sensor data interaction:

keys:
b: breathing blobs interface
c: counting interface
g: game interface
any other key: regular clock interface

mouse:
x: move mouse in x directions to simulate breathing
y: move mouse in y directions to simulate heart beat

sketch

/* 
Ghalya Alsanea
Section B
galsanea@andrew.cmu.edu
Final Project

normal mode:clock
            date
            sensor data: heart rate with beating heart

panic mode: deep breathing --- moving blob objects reacting to your breathing
            distraction technique --- use pingpong-like game to distract you
            counting technique --- count till a number to calm down
*/
// ----------------------------------------------------------------------------
// overall global variables
var proportion = 500;   //canvas proportion
var MX, MY;             //mouse x and y locations
// ----------------------------------------------------------------------------
//SENSOR global variables
var pulseSensor;        //global variable for pulse sensor data
var breathSensor;       //global variable for breath sensor data
var toggle = false;     //toggle to help sort the incoming sensor data from the serial
var pulseData = [];     //array container for incoming pulse data
// ----------------------------------------------------------------------------
// SERIAL global variables
var serial;                     // variable to hold an instance of the serialport library
var portName = 'COM11';         // fill in your serial port name here
var inData;                     // for incoming serial data. NOTE: it comes a byte at a time
var options = {baudrate: 9600}; // change the data rate to whatever you wish
// ----------------------------------------------------------------------------
// CLOCK global variables
var diameter = proportion * 0.75;   //clock diameter
//configure clock hands based on clock diameter
var secHand = diameter * 0.45;
var minHand = diameter * 0.35;
var hourHand = diameter * 0.2;
// ----------------------------------------------------------------------------
// BLOBS global variables
var blobs = [];     //used to store instances of blob objects
var change = 0;     //stores the rate of rotation and the y coordinate for noise later
var colorsPalette;  //color palette for blob objects
var numBlobs = 120; //number of blobs to make
// ----------------------------------------------------------------------------
// GAME global variables
var ballX;          //ball x position
var ballY;          //ball y position
var ballSize = 40;  //diameter of ball
var xvel;           //horizontal velocity
var yvel;           //vertical velocity
var d;              //distance between mouse and circle center
var dangerX;        //danger zone
var ballColor;      //what to color the ball
var score;          //keep score of hits
var scores = [];    //keep a list of all final scores to get the highest score
// ----------------------------------------------------------------------------
// CLICKER global variables
var gClickCount = 0;    //mouse click counter
var inc = 20;           //x increments
var yInc = 100;         //y increments         
var yPos1 = 25          //top y position
var yPos2 = yPos1 + 50; //bottom y position
// ----------------------------------------------------------------------------

function setup() {
    createCanvas(proportion, proportion);
    // setupSerial();   ..comment out to disable serial
    setupBlobs();
    setupGame();
}

// ----------------------------------------------------------------------------

function draw() {
    //constrain mouse x and y to within the canvas
    MX = constrain(mouseX, 0, width);
    MY = constrain(mouseY, 0, height);

    if (key === "b") {
        //draw the blobs
        displayBlobs();
    } else if (key === "g") {
        //draw the game
        drawGame();
    } else if (key === "c") {
        //draw clicker
        displayClicker();
    } else {
        clockDisplay();
    }
}

////////////////////////////////////////////////////////////////////////
//**************************** SERIAL ********************************//
////////////////////////////////////////////////////////////////////////

function setupSerial() {
    serial = new p5.SerialPort();       // make a new instance of the serialport library
    serial.on('data', serialEvent);     // callback for when new data arrives
    serial.open(portName);              // open the serial port
}

function serialEvent() {
    // read a byte from the serial port, convert it to a number
    // NOTE: the inData comes in a byte at a time (so range is 0-255)
    inData = serial.read();
    // sort the data incoming by distributing every other byte to their respective variable
    // serial data comes in 2 at a time, so using a toggle you can store the data correctly
    if (toggle) {
        pulseSensor = inData;
    } else {
        breathSensor = inData;
    }
    toggle = !toggle;
}


////////////////////////////////////////////////////////////////////////
//*********************** base/CLOCK Display *************************//
////////////////////////////////////////////////////////////////////////

function clockDisplay() {
    background(255, 228, 225);    //mistyRose
    noStroke();

    //show the date
    dateDisplay();

    //for testing: using mouse data instead of sensor data
    var size = map(MY, 0, height, 20, 40);
    var BPM = 88;

    // // for serial: map heart size to pulse sensor data
    // var size = map(pulseSensor, 0, 255, 20, 40);
    // 
    // //calculate beats per min (BPM) based on average pulse data
    // pulseData.push(pulseSensor);
    // //total pulse values read
    // var total = 0;
    // //find the total of all the readings
    // for(var i = 0; i < pulseData.length; i++) {
    //     total += pulseData[i];
    // }
    // //divide by the length of the data to get the average
    // var avg = total / pulseData.length;
    // //map the average number onto a range of BPM hear rates
    // var BPM = map(avg, 0, 255, 70, 130);
    //
    // //only read the last 100 readings
    // if (pulseData.length > 100) {
    //     pulseData.shift();
    // }

    //show the heart rate data
    heartDisplay(size, BPM);

    //draw from the center of the canvas
    push();
    translate(width / 2, height / 2);

    //draw clock base
    fill(255);
    circle(0, 0, diameter);

    //convert time to angles
    //subtract half a circle so the clock starts at the top, rather than 6 o'clock 
    var s = map(second(), 0, 60, 0, TWO_PI) - PI; 
    var m = map(minute(), 0, 60, 0, TWO_PI) - PI;
    //the % takes the remainder of the hours/12, which makes the dif between AM/PM 
    var h = map((hour() % 12), 0, 12, 0, TWO_PI) - PI;
 
    //draw the hands
    drawClockHands(s, secHand, 1);  //seconds hand
    drawClockHands(m, minHand, 3);  //minutes hand
    drawClockHands(h, hourHand, 5); //hour hand

    //draw the ticks every 6 degrees (6 * 60 = 360)
    for (i = 0; i < 360; i += 6) {
        //make every 5th tick thicker
        if(i % 30 == 0) {
            strokeWeight(2);
        } else {
            strokeWeight(1);
        }
        push();
        rotate(radians(i));
        line(0, diameter / 2, 0, secHand + 5);
        pop();
    }
    pop()
}

//draw the clock hands 
function drawClockHands(angle, hand, weight) {
    stroke("PaleVioletRed");
    push();
    rotate(angle);
    strokeWeight(weight);
    line(0, 0, 0, hand);
    pop();
}

//display today's date
function dateDisplay() {
    var d = day();
    var m = month();
    var y = year();
    fill("PaleVioletRed");
    textAlign(LEFT);
    textSize(18);
    text(m + "." + d + "." + y, 10, height - 15);
}

//display heart rate with heart graphic
function heartDisplay(size, BPM) {
    textSize(12);
    textAlign(CENTER);
    text(floor(BPM), width - 50, height - 60);
    drawHeart(width - 50, height - 50, size);
}

function drawHeart(x, y, size) {
    beginShape();
    vertex(x, y);
    bezierVertex(x - size / 2, y - size / 2, x - size, y + size / 3, x, y + size);
    bezierVertex(x + size, y + size / 3, x + size / 2, y - size / 2, x, y);
    endShape(CLOSE);
}

////////////////////////////////////////////////////////////////////////
//******************** breathing BLOB Display ************************//
////////////////////////////////////////////////////////////////////////

function setupBlobs() {
    //create color paletter for the blob objects
    colorsPalette = [color(146, 167, 202,30),
                    color(186, 196, 219,30),
                    color(118, 135, 172,30),
                    color(76, 41, 81,30),
                    color(144, 62, 92,30),
                    color(178, 93, 119,30),
                    color(215, 118, 136,30),
                    color(246, 156, 164,30)];

    // create new blob object for every num of blobs
    for (var i = 0; i < numBlobs; i++){
        var temp = makeBlob(i + 0.1, i, i * random(90),
            colorsPalette[floor(random(colorsPalette.length))]);
        blobs.push(temp);
    }
}

function displayBlobs() {
    background(240, 200, 240, 30);
    //for testing: using mouse data instead of sensor data
    var addToRadius = map(MX, 0, width, 0, 100);

    // //for serial: map the breathing sensor data to blob radius
    // //it's inversely mapped because when you breath out it decreases and vice versa
    // var addToRadius = map(breathSensor, 255, 0, 0, 100);

    for(var i = 0; i < blobs.length; i++){
        blobs[i].r = i + addToRadius;
        blobs[i].draw(change);
    }
    //create rotation change and noise change
    change+=0.01;
}

function makeBlob(radius, roughness, angle, color){
    var blob = {r: radius,      // radius of blob
                x: width / 2,   // x position of blob
                y: height / 2,  // y position of blob
                rough: roughness, // magnitude of how much the circle is distorted
                ang: angle,     // how much to rotate the circle by
                c: color,       // color of the blob
                draw: drawBlob, //draw the blobs
    }
    return blob;
}

function drawBlob(change) {
    noStroke(); 
    fill(this.c);   //color to fill the blob

    push();     
    translate(this.x, this.y);  //move to xpos, ypos
    rotate(this.ang + change);  //rotate by this.angle+change
    
    beginShape();   //begin a shape based on the vertex points below
    
    //create vertex points
    var off = 0;
    for (var i = 0; i < TWO_PI; i += 0.1) {
        var offset = map(noise(off, change), 0, 1, -this.rough, this.rough);
        var R = this.r + offset;
        var x = R * cos(i);
        var y = R * sin(i);
        vertex(x, y);
        off += 0.1;
    }
    endShape(); 
    pop();
}

////////////////////////////////////////////////////////////////////////
//************************ GAME display ******************************//
////////////////////////////////////////////////////////////////////////

function setupGame() {
    noStroke();
    //ball starts at the center of the canvas
    ballX = width / 2;
    ballY = height / 2;
    //define danger zone
    dangerX = width / 4;
    //set x and y velocity
    xvel = 5;
    yvel = 0;
    //set score
    score = 0;
}

function drawGame() {
    background (100);

    //draw the hitting paddle
    strokeWeight(5);
    stroke(255);
    line(mouseX, mouseY + 10, mouseX, mouseY - 10);

    //instructions
    noStroke();
    fill(255);
    textAlign(LEFT);
    writeText(20, "Let's play a game!", 20, 25);
    writeText(12, "Don't let the ball go left of the screen", 20, 50);

    //define the playing area
    fill(0);
    rect(width / 2, 0, width / 2, height);

    //warn if ball is too close to edge
    if(ballX - ballSize / 2 <= dangerX & xvel < 0){
        ballColor = "yellow"; 
    } else {
        ballColor = "white";
    }

    //draw the circle
    fill (ballColor);
    circle(ballX, ballY, ballSize);
    
    //makes the ball move in x and y directions
    ballX = ballX + xvel;
    ballY = ballY + yvel;

    //When it hits the right edge of the canvas,
    //it reverses horizontal velocity
    if (ballX >= width - ballSize / 2) {
        xvel = -1 * xvel;
    }

    //when it hits the bottom or top edge of the canvas,
    //it reverses vertical velocity
    if (ballY <= ballSize / 2) {
        yvel = -1 * yvel;
    } else if (ballY >= height - ballSize / 2) {
        yvel = -1 * yvel;
    }

    //find the distance between ball center and mouse
    d = dist(mouseX, mouseY, ballX, ballY);

    //if  (1) the mouse hits the edge of the ball 
    //and (2) the mouse is on the left half of the canvas
    //and (3) the x velocity is negative
    //then switch directions and get a random yvel. 
    if (d <= ballSize / 2 & mouseX <= width / 2 && xvel < 0){
        yvel = random(-3, 3);
        xvel = -1 * xvel;
        yvel = -1 * yvel;
        score = score + 1;            
    }

    //keep current score
    fill("white");
    writeText(30, score, width - 50, height - 50);

    //reset game if th eball went to far out the canvas
    if (ballX < -150) {
        resetGame();
    }
}

//reset game
function resetGame() {
    scores.push(score)          //push final score to scores
    background (0, 51, 102);
    fill(255);
    textAlign(LEFT);
    writeText(80, "IT'S OK!", 10, height / 4);
    writeText(30, "click to try again", 20, height / 2);
    //cureent score
    writeText(20, "score: " + score, 20, height / 2 + 50);
    //highest score
    writeText(20, "best score: " + max(scores), 20, height / 2 + 100);
    
    if (mouseIsPressed){
        setupGame();
    }
}

function writeText(size, words, x, y) {
    textSize(size);
    text(words, x, y);
}

////////////////////////////////////////////////////////////////////////
//*********************** COUNT display ******************************//
////////////////////////////////////////////////////////////////////////

function displayClicker() {
    background("pink");

    if (gClickCount === 0) {
        noStroke();
        fill(0);
        textAlign(LEFT);
        textSize(20);
        text("tap me and count to fives...", 20, 50);
    } 

    stroke (0);
    strokeWeight(3);

    //everytime you click, draw a line
    for(var i = 0; i < gClickCount; i++){
        //variable to show which line the tickmarks are on
        var a = floor((i * inc + inc - 1) / width);

        //x and y locations depending on which line
        var x = i * inc + inc - width * a;
        var y1 = yPos1 + a * yInc;
        var y2 = yPos2 + a * yInc;

        if ( (i + 1) % 5 === 0) {
            //draw diagonal line every fifth tick
            line(x, y1, x - 5 * inc, y2);
        } else {
            //otherwise draw a straight line
            line(x, y1, x, y2);
        }    
    }
}


function mousePressed() {
    //everytime you click, you increase the amount of times the loop runs
    gClickCount++;
}

function keyPressed() {
    //reset gClickCount to zero whenever a keyboard key is pressed
    gClickCount = 0;
}