An interactive music box

An wireless music player with physical interaction to minimize distraction.


Why a music box?

I like to put music on while i’m working, however in this digital age, the “recommendations” and changing between screens can be quite a distraction. A music box with physical interaction should be able to minize that as much as possible, while still kept the interaction entertaining.

How it works:

The music box would come with figurines with RFID tags. When different figurines are put on top of the music box,  different sets of music would play. Ideally, if it can be achieved, the user can put their own music album and make their own figurine.



The final project would be based on the previous assignment where I used touch sensor and Arduino Uno with a simple p5js interface. Here is the basic outline of the project:

  1. Incorporate/test RFID onto the project
  2. Connect speaker to the box instead of using laptop
  3. Refine the music library
  4. Refine the p5js interface
  5. Add the feature where user are able to make their own input
  6. Switch Arduino Uno to a Particle Photon
  7. Add lipo battery and battery shield/battery baby sitter
  8. Make a shell with laser cutted plywood


  1. Arduino Uno/Particle Photon
  2. RFID Reader chip and RFID tags
  3. Wifi base (I might be able to just use my laptop)
  4. Plywood for laser cutted bounding box
  5. Music Pieces (20s-30s per sample)
  6. Lipo battery
  7. Battery shield/baby sitter
  8. Some LEDs for indication
  9. Some buttons for play buttons etc.
  10. A cheap speaker should workout


On the day of the show, I would need a outlet for charging the battery, and a wifi base, which potentially could be my laptop. The speaker won’t be too loud so it should be ok to be somewhat close to other projects.


I probably would put up a slide show on my laptop or some print out handouts/flyers

Story – Music Box

I listen to a lot of musicals, and that is my background music when I work. Listen to the same thing over and over again is nice to some extent, however, stories might be told differently from different character’s perspective. What if the version of the story is from Elisa Schulyer instead of Alexander Hamilton? What if Cosette is the major character? Not that the music need to change (for a new show, probably).

It would be really interesting to make a interactive music box that have the same song but tells the same story from a different sequence based on the main character that you can choose. It should be mobile, with a rechargable battery, like a real music box (it would be a interesting gift). It doesn’t have to be just musical, it can get connected to wifi and search for certain artist that you wanna listen to. All you need to do is to put a figurine on top and click the play button.

What would be the advantage of having a music box than just use your phone or your computer? Well, say that you are using some music software like spotify. When you want to change a song, you have to go to that interface and click the music you want to listen to. Opening up another page could be really distracting, spotify might also recommend you this and that which doesn’t help. Having a physical interaction can potentially make you less distracted cause there is no “recommend based on history” at least.



Final project storyboarding – storytelling musicbox

The general idea of interaction goes like this:

  1. place some figurine on top of the box, assume the sequence being figurine1 (f1 for short),f2,f3……
  2. Hit play button to play music (where all figurine is a part of, from the very first story)
  3. remove some figurine and add more figurine or do nothing to play more music:
    1. if nothing change, go to the next song that all of them are in follow the story sequence
    2. if new figurine added, play the next song they all in
    3. if figurine got removed: (assume f1, f2,f3 was there)
      1. if f2 or f3 got removed, same as new figurine added
      2. if f1 got removed, f2 becomes the pivot*.


  • *Pivot in this case is that favour the pivot figurine, if no songs are really up to standards(eg: strictly solo/duet).


Virtural “Etch a Sketch”

For this assignment, I used two potentiometers to simulate the “Etch A Sketch” (Wikipedia_”Etch A Sketch”). Using two potentiometer, one controls left and right movement, and the other controls the up and down movement, the browser window is going to serve as a canvas for the “Etch A Sketch” board.


  • Parts list:
    • Potentiometer (10K)  x2
    • Adafruit M0 Metro Express
    • Bunch of jumper wires



Fritzing Sketch:


Code (Arduino Part):

const int leftRightPot = A4; //with green LED
const int upDownPot = A5; // with blue LED

const int tol = 100;

int leftRightVal = 0;
int upDownVal = 0;

int valHi = 1024;
int valLo = 0;

//Setup for Arduino to run
void setup() {
 // put your setup code here, to run once:
 pinMode(leftRightPot , INPUT);
 pinMode(upDownPot, OUTPUT);


//The operating loop
void loop() {
 // put your main code here, to run repeatedly:
 leftRightVal = reMapped(analogRead(leftRightPot));
 upDownVal = reMapped(analogRead(upDownPot));

// Serial.print("leftRightVal = ");
 // Serial.println(leftRightVal);

//Modified from class Note 5

//function used to remap input value into one
int reMapped(int input){
 int output = map(input,0,1023,0,255);
 return output;


Code (js part):

var cnv;
var serial;
var portName = 'COM4';
var dataRead;
 //@TODO : find out what port name it is

//Here are some global variables:
//@TODO: Figure out how to make them function in OOP
var curPosX = 0;
var curPosY = 0;
var leftRightVal = 255/2;
var upDownVal = 255/2;

const rightUpThresh = 150;
const leftDownThresh = 100;

var xPoint=[];
var yPoint=[];

// ****************
//Basic setup for serial port and canvas
function setup(){
 serial = new p5.SerialPort();
 // now set a number of callback functions for SerialPort
 serial.on('list', printList);
 serial.on('connected', serverConnected);
 serial.on('open', portOpen);
 serial.on('data', serialEvent);
 serial.on('error', serialError);
 serial.on('close', portClose);


//Here is canvas setup
 createCanvas(windowWidth, windowHeight);

var x = (windowWidth - width)/2;
 var y = (windowHeight - height)/2 ;

 background ("white");

//Put drawing functions in here
function draw() {
 // put drawing code here

//click mouse and reset the drawing point
function mouseSetPt(){
 //get the point where the mouse clicks
 var mouseClickX = mouseX;
 var mouseClickY = mouseY;

//update the point in the array
 xPoint = xPoint + [mouseClickX];
 yPoint = yPoint + [mouseClickY];

//update the current/new position to start
 curPosX = xPoint[xPoint.length-1];
 curPosY = yPoint[yPoint.length-1];


//use potentiometer to draw (only moving left/right and up/down)
function drawWithPot(){
//@TODO: find a better way to store tese values, like tuples?

//check right and left
 if (leftRightVal >= rightUpThresh){
 curPosX = curPosX + 1;
 else if (leftRightVal <= leftDownThresh){
 curPosX = curPosX - 1;

//check up and down
 if (upDownVal >= rightUpThresh){
 curPosY = curPosY + 1;
 else if (upDownVal <= leftDownThresh){
 curPosY = curPosY - 1;

//update the position list
 var prePosX = xPoint[xPoint.length-1];
 var prePosY = yPoint[yPoint.length-1];

if (curPosX != prePosX || curPosY != prePosY){
 xPoint = xPoint + [curPosX]
 yPoint = yPoint + [curPosY]


//Draw all the points that is saved in the array
function drawPts(){
 for ( i = 0; i < xPoint.length ; i++){
 var xPointPos = xPoint[i];
 var yPointPos = yPoint[i];
 point(xPointPos, yPointPos);

//All server related
function printList(portList) {
 // for (var i = 0; i < portList.length; i++) {
 // print(i + " " + portList[i]);
 // }

function serverConnected()

function portOpen()

function serialEvent(){

var strRead = serial.readStringUnil('\r\n');

if (strRead.length > 0 ) {
 var potStr = split(strRead, ',');

if (potStr.length == 4) { // ignore any tiny strings
 leftRightVal = potStr[1];
 upDownVal = potStr[3];


function serialError(err)
 print('serialError ' + err);

function portClose()


Zip -Etch A Sketch

Walking in the Dark

Background: For people who have migrane, it is terribly painful to turn on the light in the room (any light is unbearable at times) or have sound playing, talking would be painful. Even though this doesn’t mean they can’t open their eyes to see things. However, the light and sound sesitivity make it really difficult for them to walk around in the house for certain necesaties.

Therefore, my project is to help people when they have to walk in the dark, and lights cannot be used as an indicator. Instead of lights, I used buzzer as indicator. If the person is too close to the wall on the left, the buzzer on the left (stick on the person’s hand) will vibrate. Same goes to the buzzer on the right. However, if they have a pet at home, the pir sensor attached to their chest will be active and the buzzer will both buzz at a certain meter.

[Insert picture here]


Here is the code part of it:

/* Jean Zhang


const int leftDistPin = A1;
const int rightDistPin = A2;
const int pirPin = 8;

const int buzzLeft = 6;
const int buzzRight = 10;

const int critDist = 950; // somewhat arbitrary number

int leftDist = 0;
int rightDist = 0;

int buzzState = 0;
int buzzStatePrev = 0;

bool isPet = false;

void petPresent() {
 if (digitalRead(pirPin == HIGH)) {
 isPet = !isPet;

void determineState() {

if (leftDist > critDist) {
 if (rightDist > critDist) {
 buzzState = 1;
 else {
 buzzState = 2;
 else {
 if (rightDist > critDist) {
 buzzState = 3;
 else {
 buzzState = 4;

 if (isPet == true) {
 buzzState = 5;

void setup() {
 // put your setup code here, to run once:


pinMode(leftDistPin, INPUT);
 pinMode(rightDistPin, INPUT);
 pinMode(pirPin, INPUT);

pinMode(buzzLeft, OUTPUT);
 pinMode(buzzRight, OUTPUT);

 //attach Interrupt
 if (isPet == HIGH) {
 attachInterrupt( digitalPinToInterrupt (pirPin), petPresent, HIGH);

void loop() {
 // put your main code here, to run repeatedly:

//read distance and compare to critDist
 leftDist = analogRead(leftDistPin);
 rightDist = analogRead(rightDistPin);

buzzStatePrev = buzzState;

 Serial.print(" right");

if (buzzStatePrev != buzzState) {
 switch (buzzState) {
 case 1: break;
 case 2:
 digitalWrite(buzzRight, HIGH);
 case 3:
 digitalWrite(buzzLeft, HIGH);
 case 4:
 digitalWrite(buzzRight, HIGH);
 digitalWrite(buzzLeft, HIGH);
 case 5:
 digitalWrite(buzzRight, HIGH);
 digitalWrite(buzzLeft, HIGH);
 delay (500); //@TODO: write a function that does not use delay
 digitalWrite(buzzRight, LOW);
 digitalWrite(buzzLeft, LOW);
 delay (500); // Yeah, try not to use delay




Assignment 1 – Distance gamer

This is ment to be a toy for cats and dogs, where the closer they are to the toy, the brighter the LED is going to be. And when they put their paws on th button (it’s a photoresistor), the LED with stop for however long they leave their paws in place.



  • 1x  IR distance sensor
  • 1x  photoresistor
  • 1x  3.3v LED
  • 1x  10k potentiometer
  • 1x  220 resistor
  • Adafruit Metro M0 Express


Initially prototyped with 2 photoresistor instead of ir sensor to test the data (which haven’t really happen yet)


  Jean Zhang

  This is a sketch about a toy for pets (hopefully).
  This is Ver 1.0 which is a easy prototype, a proof of
  concept that the code/board/sensors works properly.

  This code is modified from the class 2 note from
  Prof. J. Eric Townsend.The lisence is as the following:

  The MIT License (MIT)

  Copyright (c) 2014 J. Eric Townsend

  Permission is hereby granted, free of charge, to any person obtaining a copy
  of this software and associated documentation files (the "Software"), to deal
  in the Software without restriction, including without limitation the rights
  to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
  copies of the Software, and to permit persons to whom the Software is
  furnished to do so, subject to the following conditions:

  The above copyright notice and this permission notice shall be included in
  all copies or substantial portions of the Software.


  This code is modified from the original code by Jean Zhang.
  Lots of thanks to Robert Zacharias and Professor Garth Zeglin for helping out with debugging and making the code to properly compile.

const int irPin = 0;
const int photoPin = 1;
const int ledPin = 8;

const int sampleSize = 16;

int irIndex = 0;
int irReadings[sampleSize];

int photoIndex = 0;
int photoReadings[sampleSize];

int distVal, photoVal, ledVal;

const int ambientPhoto = 600;
const int tolerance = 200;

void setup() {

  pinMode(irPin, INPUT);
  pinMode(photoPin, INPUT);
  pinMode(ledPin, OUTPUT);

  // put your setup code here, to run once:


void loop() {
  //Reading the values from ir and photoResistor
  int photo = analogRead(photoPin);
  photoReadings[photoIndex] = photo;
  photoIndex = (photoIndex + 1) % sampleSize;

  int irDist = analogRead(irPin);
  irReadings[irIndex] = irDist;
  irIndex = (irIndex + 1) % sampleSize;

  //Get readings for irSensors
  int *pirReadings = irReadings;
  distVal = getVal(pirReadings); // feed the pointer of the array (array location)
//  photoVal = getVal(*photoReadings);

  // console
#if 0
  Serial.print("low ");
  Serial.print(" high ");
  Serial.print(" mean ");
  Serial.print(" median ");
  digitalWrite(photoPin, LOW);

  int ledVal = map(distVal, 0, 1023, 0, 255);
  //0 (min) is based on the ambient setting

  if (abs(ambientPhoto - photoVal) > tolerance) {
    ledVal = 0;

  analogWrite(ledPin, ledVal);


//Modified from class note:
int getVal(int *readings) { //return mean for now
  int median = 0;
  int low = 1024;
  int high = 0;
  int mean = 0;
  for (int i = 0; i < sampleSize; i++) {
    mean += readings[i];
    if (readings[i] < low) { low = readings[i]; } if (readings[i] > high) {
      high = readings[i];
  mean = mean / sampleSize;
  median = readings[sampleSize / 2];

  return mean;
  //return median
//In C, a function cannot return an array

// tigoe's sort for an array
// since bubbleSort changes the array locally, the function doesn't
// need to return any value
void bubbleSort(int *readings) {
  int out, in, swapper;
  for (out = 0 ; out < sampleSize; out++) { // outer loop
    for (in = out; in < (sampleSize - 1); in++) { // inner loop if ( readings[in] > readings[in + 1] ) { // out of order?
        // swap them:
        swapper = readings[in];
        readings [in] = readings[in + 1];
        readings [(in + 1)] = swapper;




I haven’t been able to get my hardware to work at the moment (just started testing like 10min ago). I spent a lot of time trying to debug the program. I had lots of issue with C++ since I didn’t really learn how to program in C besides reading codes for reference in my Python Projects (so much less work…).

My biggest issue is with the syntax of functions and the “pointer”  in C. I wanted to make the return median/mean part into a function and use bubbleSort multiple times on different sensor values, where instead of changing the global, it takes in an array and sort that array. The problem being when defining function,  it is defining certain type of data/value. And the take-in value need to be defined for their data type . However, when the take in value is an array, the pointer concept comes into play, and I keep getting the error like this

invalid types 'int[int]' for array subscript

After lots of Google-ing and reading comments I start to understand the concept of pointers and modify my code accordingly. (It is actually a really simple solution, kinda like the x.^x – the dot syntax in MATLAB). Instead of using :

void bubbleSort(int readings) { ........

Simply change the expression into:

void bubbleSort(int *readings) { ........ 

Note the different between “int readings” and “int *readings“.  Where *readings is a pointer and readings is not really a valid input.

I also tried to return an array for bubbleSort, which I don’t have to do that (bad habit writing python programs, it does so much for you).  In C, a function cannot return an array, instead, the way to return arraies is to use pointers, which also gave me a lot of problems when compiling.


Lots of thanks to Robert Zacharias and Professor Garth Zeglin for helping with the bug in the code (mainly for the pointer spaghetti) and making the code compile properly.