vkamath@andrew.cmu.edu – Making Things Interactive

Room Occupancy detection to slack

vkamath@andrew.cmu.edu — Sat, 19 May 2018 01:23:51 +0000

Abstract: A slackbot that notifies a Slack channel when the room is occupied based on a simple light sensor.

Code: https://drive.google.com/open?id=139H6NqCLxnCcSYHNwvYoITcjAqYegCBE

Refernces: A simple light sensor code and P5serial [From my assignment 3] https://courses.ideate.cmu.edu/48-339/s2018/?p=252

Final Project: Emotionally Intelligent Home Assistant

vkamath@andrew.cmu.edu — Sat, 19 May 2018 01:15:33 +0000

Abstract: Existing home assistants improve interaction in the home by using speech recognition and intelligent natural language processing. However, they lack the emotional intelligence needed to listen, motivate and support the user and provide support towards a healthy mind. My project proposes the use of a custom google assistant through AIY voice kit to detect emotion, provide social support and generate empathy in the user.

The project builds on existing technology and can potentially replace the current Google home assistant.

Hardware:

AIY Voice kit and a mobile browser

Systems Diagram:

DEMO:

References

Project Proposal: An assistant for a healthy mind

vkamath@andrew.cmu.edu — Tue, 10 Apr 2018 02:50:46 +0000

Title: An assistant to promote a healthy mind.

Existing home assistants improve interaction in the home by using speech recognition and intelligent natural language processing. However, they lack the emotional intelligence needed to listen, motivate and support the user and provide support towards a healthy mind. My project proposes the use of a custom google assistant through AIY voice kit to detect emotion, provide social support and generate empathy in the user.

Proposed plan

Build custom Google home
Provide an avatar to assistant through the hologram
Build custom intents to listen to music, contact social circles actively and passively.
Detect emotion from voice or NLP
Detect user and location (optional)
A private dashboard for others to monitor the mental health of the user. (optional)

MATERIALS NEEDED

Old phone/display for assistant
Laser-cut transparent plastic
AIY Voice kit
Hardware neopixels to display assistant mood. (optional)

DELIVERABLES FOR SHOW

Half a table.

MEDIA FOR SHOW

Slideshow on a laptop

Touchy: A Game for Toddlers

vkamath@andrew.cmu.edu — Fri, 30 Mar 2018 03:51:56 +0000

Abstract: Touchy a game for toddlers.

Method: Ethnographic study, Participatory Design with Parents

I spent a few months getting to know the participant, babysitting the kid. This helped me learn the interactions that the user was familiar with. I used the design elements that player was familiar with for e.g. Hey Dugee.

Playtesting: A demo of a player using the interface.

Parent Feedback:

Next steps: Add sound, use the yellow car a toy that

Demo: http://kmarkiv.com/a5/A5js/

[update]

After seeking feedback from parents, I added a button to support to direct the child’s attention to the screen.

Demo 2: http://kmarkiv.com/a5/A5jsv2/
Code: https://drive.google.com/open?id=1YBk1J4fUhweWXnX1DdudYCCycPalWCRn
Feedback: Parents have now asked for learning component to make the child’s time productive. One design direction would be to speak the numbers slowly as the character jumps over the car.

Seminar Talk Thursday, April 5 at 1pm in GHC 6115:

vkamath@andrew.cmu.edu — Thu, 29 Mar 2018 18:15:09 +0000

Design for Autonomy and Fairness of New Technology Users in the Global South

Abstract:

2017 saw half the world online. As technology penetration and ecosystem maturity increase, there is a growing intent to use technology for socio-economic development for new technology users. However, complex long-standing challenges like affordability, safety, and socio-religious diktats affect people at the cusp of the Internet. My work aims to empower new technology users with increased autonomy and fairness through technology. I present my prior work on (i) design and evaluation of a cost transparency tool intended to help new mobile Internet users; (ii) design to tackle abuse and safety vectors for women in Internet technologies; and iii) design and deployment of an information broadcasting system for urban sex workers in India. I show how prevailing HCI assumptions of privacy, trust, and user identities need to be challenged as Internet advances to reach all edges of human society. Through these projects, I show how large problems can be practically addressed through a combination of design, policy, and algorithms.

Bio:

Sambasivan is a researcher focused on technology design for social, economic and political benefits in the Global South. Her research spans the areas of HCI and ICTD, and has won several recognitions at top conferences. She has been a researcher at Google since 2012, where she has co-founded a group to conduct future facing research on under-represented topics, such as gender equity and new technology users. Her research has influenced several large-scale real-world projects for the next billion users, and has been directly translated to core libraries, metrics, and guidance for Android and Web developers at Build for Billions, design.google/nbu, and Google I/O talks. Nithya has a Ph.D. and MS in information and computer sciences from University of California, Irvine and an MS in Human Computer Interaction (HCI) from Georgia Tech. Her dissertation focussed on technology design for the low-income communities of slums, urban sex workers and microenterprises in India. She is a recipient of Google’s Anita Borg and UC Irvine Dean’s fellowships. She has interned at Microsoft Research India, Nokia Research Center and IBM TJ Watson.

Nithya Sambasivan

HCII Seminar: Thursday, April 5 at 1pm in GHC 6115

03/22 Assignment: Pepper’s cone.

vkamath@andrew.cmu.edu — Thu, 22 Mar 2018 16:18:17 +0000

The idea is to use the mirror illusion to create a 3D animation. This is a perspective mapping project that uses a unity plugin to create 3D objects by modifying the models. This is inspired by videos on Youtube about creating holograms through the phone. It uses a coin to calibrate the location and additionally uses environment mapping techniques to create the optical map. The map is then invested to create the model through the unity code. They propose that this system can be used to create 3D animations such as virtual agents.

References

Paper: http://roxanneluo.github.io/PeppersCone.html

Talk

Assignment 7

vkamath@andrew.cmu.edu — Tue, 20 Mar 2018 18:00:19 +0000

This is a custom Google Home with a Japanese tutor. It uses a Google AIY Voice Kit and a simple Arduino script to control a servo.

Code (see readme) : https://drive.google.com/drive/folders/1oWowTgch1Pwi7Ge95i0tyf_rBSPKiHOe?usp=sharing

References

https://aiyprojects.withgoogle.com/voice

Assignment 6: Mouse motion based on wire protocol

vkamath@andrew.cmu.edu — Tue, 27 Feb 2018 19:22:00 +0000

Move an object based on accelerometer movement. I used an ADXL362 and the wire protocol to move an object on the screen. The accelerometer sends the raw XYZ values and a state of a button, the P5Js scripts map the raw acceleration values to areas on the screen. Todo: Need to update the p5js script … Continue reading

A3: Serial Port in P5

vkamath@andrew.cmu.edu — Mon, 05 Feb 2018 19:53:11 +0000

Assignment 2: LED Color Picker

The code is same as A2 but the debugging code is modified to send RGB data through the serial port, I use this information to change the color of the ellipse and show debug information on the browser.

A3 Code and JS

Reflection: Can use the joystick movement to make an interactive widget.

References: https://github.com/vanevery/p5.serialport/tree/master/examples

Assignment 2: LED Color Picker

vkamath@andrew.cmu.edu — Tue, 30 Jan 2018 04:43:31 +0000

The circuit shown above is a 3 pin led connected to a joystick. It uses the JoyStick movement to change the RGB values and the button press in the joystick is used to switch between states. The system has 5 states

All off
Red constant, change G and B by moving the joystick
Green constant, change R and G by moving the joystick
Blue constant, change others through joystick
All on.

Reflection: if I were to redo this, I would transfer the states from 2, 3 and 4 making it a real color picker, and I also think the joystick button may not be ideal for an interrupt.

Code and Files: A2