Problem:
As we transition into a working from home setup during covid, we lose alot of the immediate feedback through body language and facial cues from people we are communicating with. In particular, this is pronounced in a presenting information to a large group of people, where a successful presentation relies on tailoring the speed, complexity and cadence to engage the audience sufficiently to keep them interested, but not go too fast that you leave them behind.

When presenting a deck through online meeting platforms, presenters are often unable to monitor the real-time vibe of the audience since participants are not always visible due to screen real-estate (assuming that they have their cameras on). Even with the hand-raise feature, it’s hard for presenters to respond to questions in a timely manner.

Solution:
I identified two main modes of feedback that are most important to a presenter:

1. 1. Are there any questions?
   2. Are people paying attention?

Raising your hand
To indicate if the audience has any questions, I used a servo as a physical indicator of the presence of questions. Typically, when participants raise their hands during a meeting virtually, this is missed. By mapping this state to the servo, presenters now have a better understanding of when exactly questions are being raised.

Paying attention
One of the indicators of restlessness in your audience is fidgeting. If people are bored and unable to pay attention, they are likely to be crossing their arms, rearranging their seating posture, etc. We can monitor this by observing the rate of change in their posture. Using the poseNet in p5.js, we can abstract this information by drawing keypoints of your posture in the webcam and do a comparison.

I wasn’t quite able to get the serial output from p5.js to work, so I replaced it with a potentiometer as an abstraction for the demo 🙁

Kinetic feedback
By translating this to a tapping pattern if the changes exceed a particular threshold, presenters are able to understand when their audience get restless.

It is hard to discern the nuances if the tapping pattern were to change incrementally, so I set up 3 states with a threshold rate of change that matches 3 different tapping states.

1 No need to worry
Some movement is to be expected in the audience, so presenters don’t need to be concerned about a non-zero level of movement.

2 Antsy
When the audience starts getting restless, the tapping starts with a lower frequency. The presenter can quickly re-engage the audience by speeding up and moving on. As the audience returns back to normal and pays attention, the tapping should cease.

3 I can’t stand this anymore
If the audience crosses the second threshold indicating restlessness, the frequency of taps doubles and conveys to the presenter that they should try to wrap up the presentation ASAP since they’ve likely already lost their audience.

Schematic

PROBLEM: People experiencing finger -specific motor disabilities are often asked to participate in rehabilitation sessions, either within clinical settings or at home. Unfortunately, a lot of them (including me) do not execute the instructed exercises for finger strengthening and as a result their traumatised fingers may not fully recover back to their original potential.

SOLUTION: RehabActuators constitute a portable, soft robotic wearable that exploits tangible interaction to motivate patients to execute finger exercises like bending/unbending. RehabActuators turns rehabilitation into a playful activity, where the participant can manipulate its own finger by using interacting with the interface.

Computational Logic:

Told Design & Fabrication: 

Mechanism Overview:

Control Manipulation:

Interaction Demonstration: https://drive.google.com/drive/folders/1ldiV7o1EZ_QfcC9cEiDaGpcplat2HtpA?usp=sharing 

Arduino + Motor Shield REV3

Arduino Wiring:

-Copper tape for the two capacitive sensors

-Peristaltic liquid pump

Arduino Code:

# include <CapacitiveSensor.h>

// Capacitive Sensors
CapacitiveSensor csLeft = CapacitiveSensor(6, 7);
bool csLeftTouched = true;
long csLeftVal;

CapacitiveSensor csRight = CapacitiveSensor(6, 5);
bool csRightTouched = true;
long csRightVal;

// water pump
const int pumpPin = 13; const int speedPin = 11; const int brakePin = 8;
const int cw = HIGH;
const int ccw = LOW;

void setup() {
  //Serial.begin(9600);
  pinMode(pumpPin, OUTPUT); pinMode(speedPin, OUTPUT); pinMode(brakePin, OUTPUT);

}

void loop() {

  capacitiveSensorLeft();
  capacitiveSensorRight();
  activatePump();

}

void capacitiveSensorLeft() {

  csLeftVal = csLeft.capacitiveSensor(80); // 80: resolution
  if (csLeftVal > 1000) {
    csLeftTouched = true;
    //Serial.println("left on");
  } else if (csLeftVal < 100) {
    csLeftTouched = false;
    //Serial.println("left off");
  }
}

void capacitiveSensorRight() {

  csRightVal = csRight.capacitiveSensor(80); // resolution
  if (csRightVal > 1000) {
    csRightTouched = true;
    //Serial.println("right on");
  } else if (csRightVal < 100) {
    csRightTouched = false;
    //Serial.println("right off");
  }

}

void activatePump() {

  while (csRightTouched == false && csLeftTouched == false) {
    digitalWrite(brakePin, HIGH);
  }

  while (csRightTouched == true && csLeftTouched == false) {
    digitalWrite(brakePin, LOW);
    digitalWrite(pumpPin, cw);
    analogWrite(speedPin, 200); // 200 is the rotation speed
  }

  while (csRightTouched == false && csLeftTouched == true) {
    digitalWrite(brakePin, LOW);
    digitalWrite(pumpPin, ccw);
    analogWrite(speedPin, 200); // 200 is the rotation speed
  }

}

At the beginning of the semester, Chloe had an idea for an alarm clock that wakes people up through lights rather than sounds. I then bought an alarm clock, sunrise alarm clock, that has a similar functionality, and it has been great. Then I thought, well instead of using artificial lights, why can’t we use natural lights? (Of course natural light would not work if you are waking up in the middle of the night).

I then came up with this idea of a smart curtain that is basically a reverse sunrise alarm clock. It serves as an “alarm clock” as it opens and lets lights in at the time you set. I also added a natural light mode: curtains are closed when the sun is down, and the curtain is opened when the sun is up. Houses/apartments in China are much closer to one another than ones in the US, so you have to close your curtains at night. But I rely on the lights to help me wake up in the morning, so ever since I came back home in China, my mom has been yelling at me to close my curtains completely every other night.

I don’t have access to a linear stepper motor, nor a time module. So I used a servo motor to show the actions of opening and closing the curtains, and hardcoded the current time.

In addition, I used a potentiometer to adjust time, and I kept on getting connection issues with the potentiometer to have consistent readings. I used a push button to change mode, a photo cell to detect day/night(in practice, this photo cell needs to be placed such that only outdoor lightings are sensed but not indoor lightings), and an LCD to display mode and time.

This smart curtain can be improved in some ways. In alarm mode, the curtain could be slowly opening instead of opening at once, to fake the effect of sunrise. Like the sunrise alarm clock that can be controlled by smart phones, smart curtain should do that too and maybe even integrate with a traditional alarm system, i.e. slowly opens up at curtain and then alarm rings.

Code:

#include <LiquidCrystal.h>
#include <Servo.h>

LiquidCrystal lcd = LiquidCrystal(7,6,5,4,3,2);
Servo myservo;
const int POTPIN = A1;
const int BUTTONPIN = 13;
const int PHOTOPIN = A0;
const int SERVOPIN = 9;
const int PHOTOTHRESH = 300;


bool isAlarm = true;
bool isOpen = false;
int clockTime = 0; // used for demo purpose
long long lastLCDTime = 0; 
long long LCDinterval = 100;
long long lastDebounceTime = 0;  
long long debounceDelay = 50; 


void openCurtain() {
  for (int pos = 89; pos > 0; pos -= 1) {
    // in steps of 1 degree
    myservo.write(pos);
    delay(15);
  }
  isOpen = true;
}

void closeCurtain() {
  for (int pos = 0; pos < 90; pos += 1) {
    // in steps of 1 degree
    myservo.write(pos);
    delay(15);
  }
  isOpen = false;
}

void updateLCD(long long currTime, int h, int m, bool isAlarm) {
  if (currTime - lastLCDTime > LCDinterval) {
    lcd.clear();
    lcd.setCursor(2, 0);
    String mode = isAlarm ? "alarm" : "natural";
    lcd.setCursor(2, 1);
    lastLCDTime = currTime;
  }
}

void setup() {
  lcd.begin(16, 2);
  analogWrite(A5, 40);
  myservo.attach(SERVOPIN);
  myservo.write(0);
  pinMode(BUTTONPIN, INPUT_PULLUP);
  Serial.begin(9600);
}

void loop() {
  // read button
  long long currTime = millis();
  if (currTime-lastDebounceTime>debounceDelay) {
    if (digitalRead(BUTTONPIN) == 0) {
        isAlarm = !isAlarm;
    }
    lastDebounceTime = currTime;
  }

  // read potentiometer
  int potVal = analogRead(POTPIN);
  int potTime = potVal/(1023.0/(24.0*60.0));
  int h = potTime/60;
  int m = potTime%60;

  // read photo resistor
  int photoVal = analogRead(PHOTOPIN);

  updateLCD(currTime, h, m, isAlarm);
  if ((!isAlarm) && (!isOpen) && (photoVal>PHOTOTHRESH)) {
    openCurtain();
  }
  else if (isAlarm && (!isOpen) && (potTime==clockTime)) {
    openCurtain();
  }
  else if (isOpen && (photoVal<PHOTOTHRESH)) {
    closeCurtain();
  }
  
}

Kinetic Crit, due 15 Oct 2020

Note: 13 Oct is a work day in A10, I will be on zoom if anyone has questions.

Requirements

Combine inputs, kinetic outputs, and state machines to create a physically interactive system that changes interaction based on inputs and logic a person cannot perceive.  That is, information we can’t see, or we cannot see visible information.

Doorbell example

One example I gave early this semester was a “doorbell” for someone who cannot hear.

Inputs: doorbell, physical knock, person detector

Interaction: use inputs to determine output.  Doorbell + no person detected means someone rang the bell and walked away, was this a UPS/FedEx delivery?  Knock and person is there, is someone coming to visit?  To sell a product?  “Secret” knock pattern used by friends and a person is there, one of your friends has come to visit.

Output: Create appropriate output for the results of the interaction process.  UPS/FedEx drop off is lower priority than a friend coming for a visit.