I was considering a tombstone music box similar to the jewelry boxes with spinning toys inside.

I also was thinking of an event simulator where placing dolls on a board would place them in a simulation in unity. I also wanted to use a joystick to create a player controller for a model I animated with maya.

My friend Jackson made the 3 songs used after I gave him some small prompts, I was happy to get to include a piece of him in this odd little project.

The initial idea was a barber shop that caught on fire, not sure how well that was reflected but I was happy with the mood of the finished piece.

Electronics and Structure:

Some of the fabrication:

Some early sketches and plans:

Electronics:

arduino uno, 7408 quad AND gate, 7404 NOT gate, L293D motor driver, DFPlayer mini, 2 hobby gear motors, 1 continuous rotation servo, micro sd card, micro sd card reader/writer, 3 pushbuttons, 2 potentiometers, speakers, neo pixel ring, 4 yellow LEDs, 8 red LEDs, wires, resistor, breadboard, 5v power source

Materials:

wood, cardstock, cereal boxes, sewing pins, paperclips, wire, plastic bags, old clothing/fabric, thread, tissues, electrical tape, gaffer tape, duct tape, staples, boba straw, pebbles, Styrofoam

Tools:

soldering iron and fan, scissors, wire cutters, needle, pliers, precision knife, compass (drawing tool), stapler, pencils/pens, ruler, cutting mat

Code:

Everything but music:

#include "Arduino.h"
#include "SoftwareSerial.h"
#include "DFRobotDFPlayerMini.h"

#include <Adafruit_NeoPixel.h>
#include <Servo.h>

#define PIN            9
#define NUMPIXELS      12

SoftwareSerial mySoftwareSerial(10, 11); // RX, TX
DFRobotDFPlayerMini myDFPlayer;
void printDetail(uint8_t type, int value);

bool stage1 = false;
bool stage2 = false;
bool stage3 = false;

bool button1 = false;
bool button2 = false;
bool button3 = false;

bool song1 = false;
bool song2 = false;
bool song3 = false;

int value;
int angle;

Adafruit_NeoPixel pixels = Adafruit_NeoPixel(NUMPIXELS, PIN, NEO_GRB + NEO_KHZ800);

uint32_t red = pixels.Color(250,0,0);

Servo myservo;

void setup() {
  pixels.begin(); //backlight 
  pixels.setBrightness(0); 
  pixels.show();         

  pinMode(5, OUTPUT); //m1
  pinMode(6, OUTPUT); //m2
  myservo.attach(3); //m3
  
  pinMode(7, INPUT); //button1
  pinMode(8, INPUT); //button2
  pinMode(4, INPUT); //button3
  
  pinMode(A0, INPUT); //p1
  pinMode(A1, INPUT); //p2

  mySoftwareSerial.begin(9600);
  Serial.begin(115200);

  Serial.println();
  Serial.println(F("DFRobot DFPlayer Mini Demo"));
  Serial.println(F("Initializing DFPlayer ... (May take 3~5 seconds)"));

  if (!myDFPlayer.begin(mySoftwareSerial)) {  //Use softwareSerial to communicate with mp3.
    Serial.println(F("Unable to begin:"));
    Serial.println(F("1.Please recheck the connection!"));
    Serial.println(F("2.Please insert the SD card!"));
    while(true);
  }
  Serial.println(F("DFPlayer Mini online."));

  myDFPlayer.volume(30);  //Set volume value. From 0 to 30
}

void loop() {

  //part 1
  if (digitalRead(7) == HIGH) {
    analogWrite(5, analogRead(A0)/4);
    button1 = true; 
  } else {
    button1 = false;
  }

  //part 2
  if (digitalRead(8) == HIGH) {
    analogWrite(6, analogRead(A0)/4);
    button2 = true;
  } else {
    button2 = false;
  }

  //part 3
  if (digitalRead(4) == HIGH) {
    //m3
    value = analogRead(A1);
    angle = map(value, 0, 1023, 10, 170); 
    myservo.write(angle); 
    //backlight
    pixels.setBrightness(150);
    pixels.fill(red);
    pixels.show();
    button3 = true;
  } else {
    myservo.write(10);
    pixels.setBrightness(0);
    pixels.show();
    button3 = false;
  }

  //dfmini 
  stage1 = (button1 && !button2);
  stage2 = (button1 && button2 && !button3);
  stage3 = (button1 && button2 && button3);
  
  if (stage1 && !stage2 && !stage3 && !song1) {
    myDFPlayer.play(3);
    song1 = true;
    song2 = false;
    song3 = false;
  } else if (stage2 && !stage3 && !song2){
    myDFPlayer.play(1);
    song2 = true;
    song1 = false;
    song3 = false;
  } else if (stage3 && !song3){
    myDFPlayer.play(2);
    song3 = true;
    song1 = false;
    song2 = false;
  } 
  
  if (!stage1 && !stage2 && !stage3) {
    myDFPlayer.stop();
    song1 = false;
    song2 = false;
    song3 = false;
  }
  
}

dfmini controls:

/***************************************************
DFPlayer - A Mini MP3 Player For Arduino
 <https://www.dfrobot.com/index.php?route=product/product&product_id=1121>

 ***************************************************
 This example shows the basic function of library for DFPlayer.

 Created 2016-12-07
 By [Angelo qiao](Angelo.qiao@dfrobot.com)

 GNU Lesser General Public License.
 See <http://www.gnu.org/licenses/> for details.
 All above must be included in any redistribution
 ****************************************************/

/***********Notice and Trouble shooting***************
 1.Connection and Diagram can be found here
 <https://www.dfrobot.com/wiki/index.php/DFPlayer_Mini_SKU:DFR0299#Connection_Diagram>
 2.This code is tested on Arduino Uno, Leonardo, Mega boards.
 ****************************************************/

#include "Arduino.h"
#include "SoftwareSerial.h"
#include "DFRobotDFPlayerMini.h"

SoftwareSerial mySoftwareSerial(10, 11); // RX, TX
DFRobotDFPlayerMini myDFPlayer;
void printDetail(uint8_t type, int value);

int counter = 1;

void setup()
{
  pinMode(7, INPUT);
  
  mySoftwareSerial.begin(9600);
  Serial.begin(115200);

  Serial.println();
  Serial.println(F("DFRobot DFPlayer Mini Demo"));
  Serial.println(F("Initializing DFPlayer ... (May take 3~5 seconds)"));

  if (!myDFPlayer.begin(mySoftwareSerial)) {  //Use softwareSerial to communicate with mp3.
    Serial.println(F("Unable to begin:"));
    Serial.println(F("1.Please recheck the connection!"));
    Serial.println(F("2.Please insert the SD card!"));
    while(true);
  }
  Serial.println(F("DFPlayer Mini online."));

  myDFPlayer.volume(30);  //Set volume value. From 0 to 30
}

void loop()
{
  
  if (digitalRead(7) == HIGH) {
    if (counter == 1) {
      myDFPlayer.play(3);
    } else if (counter == 2) {
      myDFPlayer.play(1);
    } else if (counter == 3) {
      myDFPlayer.play(2);
    } else {
      myDFPlayer.stop();
    }
    counter = counter + 1;
    if (counter == 5) {
      counter = 1;
    }
    delay(200);
  }

  if (myDFPlayer.available()) {
    printDetail(myDFPlayer.readType(), myDFPlayer.read()); //Print the detail message from DFPlayer to handle different errors and states.
  }
}

void printDetail(uint8_t type, int value){
  switch (type) {
    case TimeOut:
      Serial.println(F("Time Out!"));
      break;
    case WrongStack:
      Serial.println(F("Stack Wrong!"));
      break;
    case DFPlayerCardInserted:
      Serial.println(F("Card Inserted!"));
      break;
    case DFPlayerCardRemoved:
      Serial.println(F("Card Removed!"));
      break;
    case DFPlayerCardOnline:
      Serial.println(F("Card Online!"));
      break;
    case DFPlayerPlayFinished:
      Serial.print(F("Number:"));
      Serial.print(value);
      Serial.println(F(" Play Finished!"));
      break;
    case DFPlayerError:
      Serial.print(F("DFPlayerError:"));
      switch (value) {
        case Busy:
          Serial.println(F("Card not found"));
          break;
        case Sleeping:
          Serial.println(F("Sleeping"));
          break;
        case SerialWrongStack:
          Serial.println(F("Get Wrong Stack"));
          break;
        case CheckSumNotMatch:
          Serial.println(F("Check Sum Not Match"));
          break;
        case FileIndexOut:
          Serial.println(F("File Index Out of Bound"));
          break;
        case FileMismatch:
          Serial.println(F("Cannot Find File"));
          break;
        case Advertise:
          Serial.println(F("In Advertise"));
          break;
        default:
          break;
      }
      break;
    default:
      break;
  }
}

Useful references:

DFPlayermini

https://wiki.dfrobot.com/DFPlayer_Mini_SKU_DFR0299

https://www.instructables.com/Tutorial-of-MP3-TF-16P/

L293d

https://www.tinkercad.com/things/gigwXYoyaDb

Neopixel_ring

https://www.instructables.com/3D-Printed-NeoPixel-Ring-Butterfly-With-Tinkercad/

https://www.tinkercad.com/things/c4nyZ9UpWer

Libraries found on Github:

https://github.com/adafruit/Adafruit_NeoPixel

https://github.com/DFRobot/DFRobotDFPlayerMini

TinkerCad Circuits I used to troubleshoot:

https://www.tinkercad.com/things/0evKMIHM0Ai-fantabulous-inari/editel?sharecode=NX5CYihf8vkyRuLa-rzRwghpmeox1-OVk11WzY99hrU

https://www.tinkercad.com/things/7uIwm5bLp6N-copy-of-project-1/editel?sharecode=YA4F9Lz2rxujlDz2OgQx0Xv3PKQZNSbunOs2S8PjWWM

Video:

This video shows me placing dolls on a board and pressing a button which spawns them into my Unity scene. This is only a prototype, which is far from perfect, and you can see me fiddle with the position of the blue doll (and later the red doll) after the Arduino failed to pick them up.

Description:

Place dolls on a board and push the ‘spawn’ button, the dolls on said board will then appear in a digital environment (created with Unity). Each doll can be differentiated by their color, both in their physical and digital representation. When the ‘spawn’  button is pushed, the digital environment will be emptied and then refilled with the dolls placed on the board (if any). In the digital environment, the dolls are given instructions to move to random positions inside given boundaries.

Each doll holds a different resistor that can be read by the Arduino, which is what allows the Arduino to determine the dolls color. The Arduino then sends that information to Unity, where it is interpreted and carried out.

Music: Brian Eno – New Space Music (Visualizer)

Progress Images:

These images include some of the 3d models I made as well as  parts of the fabrication process.

Reflection:

There are many events in our lives that we share with our family and friends. Under quarantine however, we are no longer able to share the same physical spaces with one another as we once could. It was my goal, for this project, to simulate these life events as they may have appeared before quarantine. Where representations of our family and friends would be able to share a virtual environment in lieu of a physical one.

I chose to create a funeral setting after attending a zoom call in memorial of my late uncle. I began to picture what my own funeral might look like, and who I hoped would attend.

So I made a board that would essentially act as the invite list, and a few dolls that would represent the people you might invite.

This project is only a prototype of the initial idea. The environment I made was very open to interpretation, and not necessarily personal. In a future iteration, I would make it so that, rather than color, each doll would represent a specific person in your life. I feel that this would drastically change how a person would interact with this project. Specifically, I believe that it would make placing the dolls on the board a much more personal experience. There are many other levels of customization I could hope to incorporate, but to create a literal connection between the dolls at your disposal and the people in your life is perhaps the most significant.

I was glad to be able to incorporate Unity in this project. Getting the Arduino to communicate with Unity was a good learning experience, and I believe will be useful in the future. I used SolidWorks to create the models in the scene as well.  It was very satisfying to be able to bring together these different skills I’ve developed over the semester into one project.

Code:

Arduino:

int pin_1 = A0;
int pin_2 = A1;
int pin_3 = A2;
int pin_4 = A3;

int sim_button = 5;

int array_size = 4;
int pin_list[] = {pin_1, pin_2, pin_3, pin_4};
float pin_readings[] = {0.0, 0.0, 0.0, 0.0};
int send_to_unity[] = {0, 0, 0, 0};

int pin; 
float pin_reading = 0.0;

int raw = 0;
int Vin = 5;
float Vout = 0;
float R_1K = 1000;
float R_unknown = 0;
float buffer = 0;


void setup() {
  Serial.begin(9600);
  
  pinMode(pin_1, INPUT);
  pinMode(pin_2, INPUT);
  pinMode(pin_3, INPUT);
  pinMode(pin_4, INPUT);
  pinMode(sim_button, INPUT);
}

void loop() {
  if (digitalRead(sim_button) == HIGH) {
    for (int i = 0; i < array_size; i++) {
      pin = pin_list[i];
      pin_reading = resistor_read(pin);
      pin_readings[i] = pin_reading;
      send_to_unity[i] = doll_num(pin_reading);
    }
    
    for (int i = 0; i < array_size; i++) {
      Serial.write(i);
      Serial.flush();
      Serial.write(send_to_unity[i]);
      Serial.flush();
    }    
    delay(1000);
  }
}

float resistor_read(int pin) {
  raw = analogRead(pin);
  if(raw){
    buffer = raw * Vin;
    Vout = (buffer)/1024.0;
    buffer = (Vin/Vout) - 1;
    R_unknown = R_1K * buffer;
    return(R_unknown);
  } else {
    return(0.0);
  }
}

int doll_num(float resistor_val) {
  if (0.0 <= resistor_val && resistor_val < 5.0) {
    //0 empty
    return 0;
  } else if (90.0 < resistor_val && resistor_val < 150.0) {
    //100 red
    return 1;
  } else if (300.0 < resistor_val && resistor_val < 500.0) {
    //330 yellow
    return 2;
  } else if (1900.0 < resistor_val && resistor_val < 2600.0) {
    //2k white
    return 3;
  } else if (4000.0 < resistor_val && resistor_val < 6000.0) {
    //5k1 blue
    return 4;
  } else {
    return 0;
  }
}

Unity:

spawn_dolls:

using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using System.IO.Ports;

public class spawn_dolls : MonoBehaviour
{
    
    public SerialPort sp = new SerialPort("/dev/cu.usbmodem14101", 9600);
    public GameObject doll_prefab;

    private int pin_num = 0;
    private int pin_name = 0;

    private GameObject[] doll_list;

    public Material color;

    void Start()
    {
        sp.Open();
        sp.ReadTimeout = 1;

        doll_list = new GameObject[] {null, null, null, null};
    }

    // Update is called once per frame
    void Update()
    {
        if (sp.IsOpen) {
            try 
            { 
                // read the incoming byte:
                pin_num = sp.ReadByte();
                pin_name = sp.ReadByte();

                if (pin_num == 0) {
                    //first doll, delete last batch
                    for (int i = 0; i < 4; i++){
                        if (doll_list[i] != null) {
                            Destroy(doll_list[i]);
                            doll_list[i] = null;
                        }
                    }
                }

                if (pin_name != 0) {
                    spawn_doll(pin_num, pin_name);
                }

            }
            catch (System.Exception)
            {

            }
        }
    }


    void spawn_doll(int pin_num, int pin_name){

        Vector3 position = new Vector3(Random.Range(-300.0F, 300.0F), 100, Random.Range(-150.0F, 200.0F));//position
        GameObject doll_clone = (GameObject) Instantiate(doll_prefab, position, Quaternion.identity);

        GameObject body = doll_clone.transform.GetChild(0).gameObject;
        GameObject hands = doll_clone.transform.GetChild(1).gameObject;
        GameObject head = doll_clone.transform.GetChild(2).GetChild(0).gameObject;

        Renderer body_render = body.GetComponent<Renderer>();
        Renderer hands_render = hands.GetComponent<Renderer>();
        Renderer head_render = head.GetComponent<Renderer>();


        if (pin_name == 1) {
            body_render.material.color = Color.red;
            hands_render.material.color = Color.red;
            head_render.material.color = Color.red;
            doll_clone.transform.localScale = new Vector3(0.9F,0.9F,0.9F);

        } else if (pin_name == 2) {
            body_render.material.color = Color.yellow;
            hands_render.material.color = Color.yellow;
            head_render.material.color = Color.yellow;
            doll_clone.transform.localScale = new Vector3(0.92F,0.92F,0.92F);

        } else if (pin_name == 3) {
            body_render.material.color = Color.white;
            hands_render.material.color = Color.white;
            head_render.material.color = Color.white;
            doll_clone.transform.localScale = new Vector3(0.94F,0.94F,0.94F);

        } else if (pin_name == 4) {
            body_render.material.color = Color.blue;
            hands_render.material.color = Color.blue;
            head_render.material.color = Color.blue;
            doll_clone.transform.localScale = new Vector3(0.96F,0.96F,0.96F);

        } else {
            body_render.material.color = Color.black;
            hands_render.material.color = Color.black;
            head_render.material.color = Color.black;
            doll_clone.transform.localScale = new Vector3(1F,1F,1F);
        }

        doll_list[pin_num] = doll_clone;
    }
}

doll_movement:

using System.Collections;
using System.Collections.Generic;
using UnityEngine;

public class doll_movement : MonoBehaviour
{   
    private GameObject head;
    private GameObject hands;
    private GameObject body;
    private GameObject head_axis;
    private GameObject neck_axis;

    public float xMax;
    public float zMax;
    public float xMin;
    public float zMin;

    public float moveSpeed = 2;
    public float turnSpeed = 45;
    
    private Vector3 position = Vector3.zero;

    private float timer;
    private float timer_limit;

    private int move_bool;

    void Start()
    {
        body  = transform.GetChild(0).gameObject;
        hands  = transform.GetChild(1).gameObject;
        head_axis  = transform.GetChild(2).gameObject;
        head  = transform.GetChild(2).GetChild(0).gameObject;
        neck_axis  = transform.GetChild(3).gameObject;

        timer_limit = Random.Range(1.0f, 10.0f);
        move_bool = Random.Range(0, 4);

        RandomizePosition();

    }

    void Update()
    {
        timer += Time.deltaTime;
        
        if (transform.localPosition.x > xMax) {
            RandomizePosition();
            timer = 0.0f;
            timer_limit = Random.Range(1.0f, 10.0f);
            move_bool = Random.Range(0, 4);
        }
        if (transform.localPosition.x < xMin) {
            RandomizePosition();
            timer = 0.0f;
            timer_limit = Random.Range(1.0f, 10.0f);
            move_bool = Random.Range(0, 4);
        }
        if (transform.localPosition.z > zMax) {
            RandomizePosition();
            timer = 0.0f;
            timer_limit = Random.Range(1.0f, 10.0f);
            move_bool = Random.Range(0, 4);
        }
        if (transform.localPosition.z < zMin) {
            RandomizePosition();
            timer = 0.0f;
            timer_limit = Random.Range(1.0f, 10.0f);
            move_bool = Random.Range(0, 4);
        }

        if (timer > timer_limit) {
            RandomizePosition();
            timer = 0.0f;
            timer_limit = Random.Range(1.0f, 10.0f);
            move_bool = Random.Range(0, 4);      
        }

        if (move_bool != 3) {
            MoveGameObject();
        }

    }

    private void RandomizePosition()
    {
        position = new Vector3(Random.Range(-200, 200),transform.position.y, Random.Range(-200, 200));
    }

    private void MoveGameObject ()
    {
        Vector3 angle = position - transform.position;
        if(angle.y != 0)angle.y = 0;
        if(position.y != transform.position.y)position.y = transform.position.y;
        if(transform.position != Vector3.MoveTowards(transform.position, position, moveSpeed * Time.deltaTime))
        {
            transform.position = Vector3.MoveTowards(transform.position, position, moveSpeed * Time.deltaTime);
            transform.rotation = Quaternion.RotateTowards(transform.rotation,Quaternion.LookRotation(angle),turnSpeed * Time.deltaTime);
        }
    }

}

Resources used:

• https://www.circuitbasics.com/arduino-ohm-meter/
• https://www.alanzucconi.com/2015/10/07/how-to-integrate-arduino-with-unity/
• https://answers.unity.com/questions/336663/random-movement-staying-in-an-area.html
• https://answers.unity.com/questions/1456945/how-to-make-a-gameobject-move-randomly-withing-a-b.html

When the flame sensor attached to Puppet A’s arm detects candlelight, Puppet A will slowly extend their tongue into Puppet B’s mouth, closing a circuit that powers the LEDs in Puppet B’s eyes. Careless Whisper will play/pause when the flame sensor is on/off. Puppet A’s eyes will also light up and their eyebrows will furrow the further their tongue extends.

Reflection:

In this time of social distancing, we lack the ability to reach out and touch one another as we once did, whether these relationships be mundane or intimate is irrelevent as they are all a vital part of our human nature, which up until now we have taken for granted.

I wanted to create an interaction between two puppets/animatronics, but didn’t want to over complicate said interaction. Ideally, I would have liked to include a conversation between the two puppets, however due to the lack of time and the scope of the assignment, I decided to move forward with a non-verbal interaction.

I considered preparing audio for this piece, but as far as I know, the dfmini is unable to play multiple mp3 files at the same time. So if I wanted both puppets to speak at once, I would have to put both voices on the same mp3 file, and manually sync that with the puppets respective mouths. I thought that it might be too much of an undertaking.

After some sketching, I landed on a funny little idea, where by extending one puppets tongue into the others mouth, the eyes of the puppet light up. This would be triggered by candlelight, with some goofy music to set the mood.

Thinking of all the obstacles people go through to create a connection, I thought it might be fun to add some more hurdles for my automaton representation.

I was moving the hand (with the flame sensor) around manually, but I could have had it triggered by some different input, and turn this into some sort of Rube Goldberg machine. For instance, the hand will only move into position after 8pm, the tongue will only extend if the hand is next to a flame, and the eyes will only light up if the puppets are near enough for the tongue to touch the copper tape in Puppet B’s mouth. Or maybe it has to be the third date, and the lighting has to be in a certain range. Just adding more reasons for things to go wrong, just like they do in real life.

It was interesting to hear others interpretations of this in class. It was not my intention to make something uncomfortable to watch, but I understand how it can be. Upon reflection, I found that, with very minor changes, these animatronics could represent much darker themes if I so choose. I think it goes to show the power of the face, and a puppets ability to mimic them. When all is said and done, it’s just some foam glued to some cardboard, but when it’s put together in a certain way, we can attach meaning and emotion to it.

Since I was experimenting with making these puppets, I thought I would play with the shapes a little more, thus the recoiled position of Puppet B. If I had some foresight, I probably would have realized that it might end up looking a little alarming.

Upon further reflection I decided it was unproductive to assign genders to the two. I don’t view either of them in any way, and in my mind, it’s up to the viewer to make those assumptions if they choose. Again, this was meant to be a lighthearted project, but I feel that under a heteronormative assumption, there are issues with the act of giving and the act of receiving, those being roles stereotypically anchored to men and women respectively.

It was a cool project, and I learned a lot. Definitely gained some perspective, specifically that I need to pay more attention to others perspectives.

Electronics:

• • 2 Arduinos (I found it easier to use a second Arduino for audio)
  • Flame sensor
  • Servo
  • Micro Servo
  • 2 red LED, 2 yellow LED
  • Dfmini
  • Speaker
  • Wires, bread boards, resistor

Materials:

• • Chipboard (cereal boxes etc.)
  • Mattress foam
  • Oil pastels
  • plastic cup
  • Lego gears, beams, axles, and pins
  • Hot glue
  • Copper tape
  • Electrical tape

Code:

Movement:

#include <Servo.h>

Servo tongue;
Servo brow;

int flame_sensor = 3;
int angle = 170;
int old_angle;

int brow_angle = 90;


void setup() {
  pinMode(flame_sensor, INPUT);
  tongue.attach(5);
  brow.attach(6);

  brow.write(brow_angle);
  tongue.write(angle);
}

void loop() {
  old_angle = angle;
  
  if (digitalRead(flame_sensor) == HIGH) { 
    if (angle < 170) {
      angle = angle + 2;
    }
  } else {
    if (angle > 10) {
        angle = angle - 2;
      }  
  }
  
  if (old_angle != angle) {
    brow_angle = map(angle, 10, 170, 45, 90);
    tongue.write(angle);
    brow.write(brow_angle);
  }
  delay(50);
}

Audio:

/***************************************************
DFPlayer - A Mini MP3 Player For Arduino
 <https://www.dfrobot.com/index.php?route=product/product&product_id=1121>

 ***************************************************
 This example shows the basic function of library for DFPlayer.

 Created 2016-12-07
 By [Angelo qiao](Angelo.qiao@dfrobot.com)

 GNU Lesser General Public License.
 See <http://www.gnu.org/licenses/> for details.
 All above must be included in any redistribution
 ****************************************************/

/***********Notice and Trouble shooting***************
 1.Connection and Diagram can be found here
 <https://www.dfrobot.com/wiki/index.php/DFPlayer_Mini_SKU:DFR0299#Connection_Diagram>
 2.This code is tested on Arduino Uno, Leonardo, Mega boards.
 ****************************************************/

#include "Arduino.h"
#include "SoftwareSerial.h"
#include "DFRobotDFPlayerMini.h"

SoftwareSerial mySoftwareSerial(10, 11); // RX, TX
DFRobotDFPlayerMini myDFPlayer;
void printDetail(uint8_t type, int value);

int flame_sensor = 3;

bool song_paused = false;

void setup()
{
  mySoftwareSerial.begin(9600);
  Serial.begin(115200);
  
  Serial.println();
  Serial.println(F("DFRobot DFPlayer Mini Demo"));
  Serial.println(F("Initializing DFPlayer ... (May take 3~5 seconds)"));
  
  if (!myDFPlayer.begin(mySoftwareSerial)) {  //Use softwareSerial to communicate with mp3.
    Serial.println(F("Unable to begin:"));
    Serial.println(F("1.Please recheck the connection!"));
    Serial.println(F("2.Please insert the SD card!"));
    while(true);
  }
  Serial.println(F("DFPlayer Mini online."));

  pinMode(flame_sensor, INPUT);
  
  myDFPlayer.volume(30);  //Set volume value. From 0 to 30
  myDFPlayer.play(1);  //Play the first mp3
}

void loop()
{
  static unsigned long timer = millis();
  if (digitalRead(flame_sensor) == LOW) { 
    if (song_paused == true) {
      myDFPlayer.start();
      song_paused = false;
    }
  } else {
    if (song_paused == false) {
      myDFPlayer.pause();
      song_paused = true;
    }
  }

  if (myDFPlayer.available()) {
    printDetail(myDFPlayer.readType(), myDFPlayer.read()); //Print the detail message from DFPlayer to handle different errors and states.
  }
}

void printDetail(uint8_t type, int value){
  switch (type) {
    case TimeOut:
      Serial.println(F("Time Out!"));
      break;
    case WrongStack:
      Serial.println(F("Stack Wrong!"));
      break;
    case DFPlayerCardInserted:
      Serial.println(F("Card Inserted!"));
      break;
    case DFPlayerCardRemoved:
      Serial.println(F("Card Removed!"));
      break;
    case DFPlayerCardOnline:
      Serial.println(F("Card Online!"));
      break;
    case DFPlayerPlayFinished:
      Serial.print(F("Number:"));
      Serial.print(value);
      Serial.println(F(" Play Finished!"));
      break;
    case DFPlayerError:
      Serial.print(F("DFPlayerError:"));
      switch (value) {
        case Busy:
          Serial.println(F("Card not found"));
          break;
        case Sleeping:
          Serial.println(F("Sleeping"));
          break;
        case SerialWrongStack:
          Serial.println(F("Get Wrong Stack"));
          break;
        case CheckSumNotMatch:
          Serial.println(F("Check Sum Not Match"));
          break;
        case FileIndexOut:
          Serial.println(F("File Index Out of Bound"));
          break;
        case FileMismatch:
          Serial.println(F("Cannot Find File"));
          break;
        case Advertise:
          Serial.println(F("In Advertise"));
          break;
        default:
          break;
      }
      break;
    default:
      break;
  }
}

Alarm Clock:

An alarm clock determined to fulfill its purpose.

Servo in the mouth “synced” up with audio (close enough). Neo pixel ring light hidden behind eyes, color changes to red and the brightness increases for each “wake up”. Eyebrows also furrow a bit further with each “wake up”. Snooze can be triggered after each “wake up”, the audio will stop, eyebrows will raise, and the eyes will turn green.

Reflection:

This was a lot of fun to make.

Took awhile to get the mouth to move with the audio in a way that I wanted. The servo had to rotate fully to open the mouth, and I realized that it wasn’t moving fast enough to fully reach a position before the next command was given. So although the open angle and closed angle are the same for each word, the servo doesn’t always have enough time to fully extend. Only the last one where the “wake up” was dragged out does the mouth fully open.  I could have moved the servo so it wouldn’t have to rotate so much to open the mouth, but I thought that it kind of worked out for the best. Also the face kind of shakes when the mouth is fully opened which makes it look a little angrier, happy accident.

I wanted to add a lot more to this, at the moment it’s not really an alarm clock, it turns on with a button and turns off with another button. But I do plan on finishing it in the future.

Ideally, I wanted to add in a real time clock, an lcd to show the current time, some simple buttons to set the alarm, which would also be shown on the lcd, a motion sensor on top to act as a snooze button, and a passive buzzer to make it a little more disruptive to your sleep. I’d also like to give the face some color, and maybe add some thin rings around the eyes.

Also also, there are a few things I’d like to debug about this as it is now. The code for the audio is run on  a second Arduino, and I think it would be for the best if it wasn’t, but because the code for the mouth/eyebrows/eyes uses the delay function, I found it a lot easier to just separate the two. Also concerning sound, every time I’ve used speakers I end up with a weird clicking noise, even when no audio is playing. You can hear it at the end of the video. I’m sure there are simple solutions for these issues, and I look forward to figuring them out.

Build wise, ideally ideally, I would laser cut the body, use a kerf cut (I think it’s called that) to make the round part, probably add some supports, laser cut acrylic for the eyes, and laser cut or 3d print gears for the eyebrows and maybe mouth. The legos gears were useful but a bit limited, I had a small range of gear sizes and the positions they could be placed in. Also, cardboard and scissors are great for prototyping, but there’s no replacing the precision of a laser cutter, and wood also happens to looks nicer. Maybe I’ll try this again now that I have the experience.

Electronics:

• • 2 Arduino (I found it easier to use a second Arduino for audio)
  • Servo
  • Micro Servo
  • Neo pixel ring
  • Dfmini
  • Speaker
  • 2 buttons
  • Wires, bread boards, resistors

Materials:

• • Chipboard (cereal boxes etc.)
  • Mattress foam
  • plastic cup
  • Lego gears, beams, axles, and pins
  • Paper towel
  • Hot glue
  • Copper tape
  • Electrical tape

Code:

Movement:

#include "Arduino.h"
#include <Servo.h>

#include <Adafruit_NeoPixel.h>

#define PIN            9
#define NUMPIXELS      12

int alarm_button = 3;
int snooze_button = 4;

int mouthPin = 7;
Servo mouth;

int openAngle = 70;
int closedAngle = 170;

int browPin = 8;
Servo brow;

int brow_1 = 80;
int brow_2 = 100;
int brow_3 = 130;
int brow_4 = 145;
int brow_5 = 160;

bool wake_up = false;
int act = 1;

Adafruit_NeoPixel pixels = Adafruit_NeoPixel(NUMPIXELS, PIN, NEO_GRB + NEO_KHZ800);

uint32_t color_1 = pixels.Color(210,250,250); //light blue
uint32_t color_2 = pixels.Color(220,150,100); 
uint32_t color_3 = pixels.Color(230,100,25); 
uint32_t color_4 = pixels.Color(250,50,0);    //orange
uint32_t color_5 = pixels.Color(250,0,0);     //red
uint32_t color_6 = pixels.Color(0,250,0);     //green

void setup() {
  Serial.begin(9600);

  pixels.begin(); 
  pixels.setBrightness(0); 
  pixels.show(); 
  
  pinMode(alarm_button, INPUT);
  pinMode(snooze_button, INPUT);
  
  mouth.attach(mouthPin);
  mouth.write(closedAngle);

  brow.attach(browPin);
  brow.write(brow_1);
}

void loop() {

  if (wake_up == true) {

    // first wake-up
    if (act == 1) {
      pixels.setBrightness(70);
      pixels.fill(color_1);
      pixels.show();

      brow.write(brow_1);
      
      delay(790);
      
      //"wake up" at ~.85 seconds
      mouth.write(openAngle);
      delay(270);
      
      //close end word
      mouth.write(closedAngle);
      delay(1850); //1780 was a real sweet spot
    }
    
    if (act == 2) {
      pixels.setBrightness(100);
      pixels.fill(color_2);
      pixels.show();

      brow.write(brow_2);
      
      //"wake up" at ~3.1 seconds
      mouth.write(openAngle);
      delay(270);
     
      //close end word
      mouth.write(closedAngle);
      delay(1460);
    }
    
    if (act == 3) {
      pixels.setBrightness(120);
      pixels.fill(color_3);
      pixels.show();

      brow.write(brow_3);
      
      //"wake up" at ~4.8 seconds
      mouth.write(openAngle);
      delay(230);
      
      //close end word
      mouth.write(closedAngle);
      delay(1400);
    }
    
    if (act == 4) {
      pixels.setBrightness(160);
      pixels.fill(color_4);
      pixels.show();

      brow.write(brow_4);
      
      //"wake" at ~6.4 seconds
      mouth.write(openAngle);
      delay(220);

      //close end word
      mouth.write(closedAngle);
      delay(1400);
    }

    if (act == 5) {
      pixels.setBrightness(180);
      pixels.fill(color_5);
      pixels.show();

      brow.write(brow_5);
      
      //"wake" 
      mouth.write(openAngle);
      delay(310);
      //close between words
      mouth.write(closedAngle);
      delay(200);
      //"up"
      mouth.write(openAngle);
      delay(700);
      //close end word
      mouth.write(closedAngle);
      delay(1500);
    }
  
    act = act + 1;
    if (act > 5) {
      act = 1;
    }

  }

  if (digitalRead(alarm_button) == HIGH) {
    wake_up = true;    
  }
  
  if (digitalRead(snooze_button) == HIGH) {
    if (wake_up == true) {
      wake_up = false;
      
      pixels.setBrightness(120);
      pixels.fill(color_6);
      pixels.show();
      
      brow.write(brow_1);
      
      delay(1000);
    }
  }
}

Audio:

/***************************************************
DFPlayer - A Mini MP3 Player For Arduino
 <https://www.dfrobot.com/index.php?route=product/product&product_id=1121>

 ***************************************************
 This example shows the basic function of library for DFPlayer.

 Created 2016-12-07
 By [Angelo qiao](Angelo.qiao@dfrobot.com)

 GNU Lesser General Public License.
 See <http://www.gnu.org/licenses/> for details.
 All above must be included in any redistribution
 ****************************************************/

/***********Notice and Trouble shooting***************
 1.Connection and Diagram can be found here
 <https://www.dfrobot.com/wiki/index.php/DFPlayer_Mini_SKU:DFR0299#Connection_Diagram>
 2.This code is tested on Arduino Uno, Leonardo, Mega boards.
 ****************************************************/

#include "Arduino.h"
#include "SoftwareSerial.h"
#include "DFRobotDFPlayerMini.h"

SoftwareSerial mySoftwareSerial(10, 11); // RX, TX
DFRobotDFPlayerMini myDFPlayer;
void printDetail(uint8_t type, int value);

int alarm_button = 2;
int snooze_button = 3;
bool song_paused = true;

void setup()
{
  mySoftwareSerial.begin(9600);
  Serial.begin(115200);
  
  Serial.println();
  Serial.println(F("DFRobot DFPlayer Mini Demo"));
  Serial.println(F("Initializing DFPlayer ... (May take 3~5 seconds)"));
  
  if (!myDFPlayer.begin(mySoftwareSerial)) {  //Use softwareSerial to communicate with mp3.
    Serial.println(F("Unable to begin:"));
    Serial.println(F("1.Please recheck the connection!"));
    Serial.println(F("2.Please insert the SD card!"));
    while(true);
  }
  Serial.println(F("DFPlayer Mini online."));

  pinMode(alarm_button, INPUT);
  pinMode(snooze_button, INPUT);
  
  myDFPlayer.volume(30);  //Set volume value. From 0 to 30
}

void loop()
{
  static unsigned long timer = millis();
  if (digitalRead(alarm_button) == HIGH) { 
    if (song_paused == true) {
      myDFPlayer.play(2);
      song_paused = false;
    }
  } 
  
  if (digitalRead(snooze_button) == HIGH) { 
    if (song_paused == false) {
      myDFPlayer.stop();
    }
  } 
  
  if (myDFPlayer.available()) {
    printDetail(myDFPlayer.readType(), myDFPlayer.read()); //Print the detail message from DFPlayer to handle different errors and states.
  }
}

void printDetail(uint8_t type, int value){
  switch (type) {
    case TimeOut:
      Serial.println(F("Time Out!"));
      break;
    case WrongStack:
      Serial.println(F("Stack Wrong!"));
      break;
    case DFPlayerCardInserted:
      Serial.println(F("Card Inserted!"));
      break;
    case DFPlayerCardRemoved:
      Serial.println(F("Card Removed!"));
      break;
    case DFPlayerCardOnline:
      Serial.println(F("Card Online!"));
      break;
    case DFPlayerPlayFinished:
      Serial.print(F("Number:"));
      Serial.print(value);
      Serial.println(F(" Play Finished!"));
      break;
    case DFPlayerError:
      Serial.print(F("DFPlayerError:"));
      switch (value) {
        case Busy:
          Serial.println(F("Card not found"));
          break;
        case Sleeping:
          Serial.println(F("Sleeping"));
          break;
        case SerialWrongStack:
          Serial.println(F("Get Wrong Stack"));
          break;
        case CheckSumNotMatch:
          Serial.println(F("Check Sum Not Match"));
          break;
        case FileIndexOut:
          Serial.println(F("File Index Out of Bound"));
          break;
        case FileMismatch:
          Serial.println(F("Cannot Find File"));
          break;
        case Advertise:
          Serial.println(F("In Advertise"));
          break;
        default:
          break;
      }
      break;
    default:
      break;
  }
}

Other:

During the process, I experimented with a couple different puppets, but eventually settled on those above. These are the other animatronics I made during this project, albeit without the heartbeat of an Arduino Uno.

Progress Pics:

found some old legos

some sketches

Materials:

• antron fleece
  • https://puppetpelts.com/collections/true-colors-collection/products/splash
• aluminum wire
  • https://www.homedepot.com/p/OOK-50-ft-10-lb-18-Gauge-Aluminum-Hobby-Wire-50176/100192917#product-overview
• polyfoam
  •  https://foamonline.com/
  • cube/rect, dry fast, 24in length, 48in width, .5in height, medium firmness
• contact cement
  • https://www.amazon.com/gp/product/B00BZXB2FI/ref=as_li_tl?ie=UTF8&tag=adamkre-20&camp=1789&creative=9325&linkCode=as2&creativeASIN=B00BZXB2FI&linkId=2064399a7cd3acf3959515713ca2defb
• ping pong balls
• 3d printed gears
• many servos and micro servos

various sensors to act as various triggers:

• sound sensor
• heat sensor
• light sensor
• sonic sensor

Resources:

super useful site, makes svg files for gears based on your input.

http://hessmer.org/gears/InvoluteSpurGearBuilder.html?circularPitch=8&pressureAngle=20&clearance=0.05&backlash=0.05&profileShift=0&gear1ToothCount=30&gear1CenterHoleDiamater=4&gear2ToothCount=8&gear2CenterHoleDiamater=4&showOption=3

very well made eye mechanisms

6 Animatronic Eye Mechanisms You Can Download and 3D Print

https://drive.google.com/file/d/1kk2RDLHNMECK64MiSgRnj1lKMDavQghA/view?usp=sharing

I wasn’t able to turn the powerpoint into a google slide because it was too large, I’m sorry for the inconvenience

Libraries found on Github:

https://github.com/adafruit/Adafruit_NeoPixel

https://github.com/DFRobot/DFRobotDFPlayerMini

Useful references:

DFPlayermini

https://wiki.dfrobot.com/DFPlayer_Mini_SKU_DFR0299

https://www.instructables.com/Tutorial-of-MP3-TF-16P/

L293d

https://www.tinkercad.com/things/gigwXYoyaDb

Neopixel_ring

https://www.instructables.com/3D-Printed-NeoPixel-Ring-Butterfly-With-Tinkercad/

https://www.tinkercad.com/things/c4nyZ9UpWer

they were just for tests so I didn’t finish them, but if you want to see, these are the links

https://www.tinkercad.com/things/0evKMIHM0Ai-fantabulous-inari/editel?sharecode=NX5CYihf8vkyRuLa-rzRwghpmeox1-OVk11WzY99hrU

https://www.tinkercad.com/things/7uIwm5bLp6N-copy-of-project-1/editel?sharecode=YA4F9Lz2rxujlDz2OgQx0Xv3PKQZNSbunOs2S8PjWWM