Abstract: Existing home assistants improve interaction in the home by using speech recognition and intelligent natural language processing. However, they lack the emotional intelligence needed to listen, motivate and support the user and provide support towards a healthy mind. My project proposes the use of a custom google assistant through AIY voice kit to detect emotion, provide social support and generate empathy in the user.
The project builds on existing technology and can potentially replace the current Google home assistant.
Existing home assistants improve interaction in the home by using speech recognition and intelligent natural language processing. However, they lack the emotional intelligence needed to listen, motivate and support the user and provide support towards a healthy mind. My project proposes the use of a custom google assistant through AIY voice kit to detect emotion, provide social support and generate empathy in the user.
Build custom Google home
Provide an avatar to assistant through the hologram
Build custom intents to listen to music, contact social circles actively and passively.
Detect emotion from voice or NLP
Detect user and location (optional)
A private dashboard for others to monitor the mental health of the user. (optional)
Old phone/display for assistant
Laser-cut transparent plastic
AIY Voice kit
Hardware neopixels to display assistant mood. (optional)
Method: Ethnographic study, Participatory Design with Parents
I spent a few months getting to know the participant, babysitting the kid. This helped me learn the interactions that the user was familiar with. I used the design elements that player was familiar with for e.g. Hey Dugee.
Playtesting: A demo of a player using the interface.
Next steps: Add sound, use the yellow car a toy that
Design for Autonomy and Fairness of New Technology Users in the Global South
2017 saw half the world online. As technology penetration and ecosystem maturity increase, there is a growing intent to use technology for socio-economic development for new technology users. However, complex long-standing challenges like affordability, safety, and socio-religious diktats affect people at the cusp of the Internet. My work aims to empower new technology users with increased autonomy and fairness through technology. I present my prior work on (i) design and evaluation of a cost transparency tool intended to help new mobile Internet users; (ii) design to tackle abuse and safety vectors for women in Internet technologies; and iii) design and deployment of an information broadcasting system for urban sex workers in India. I show how prevailing HCI assumptions of privacy, trust, and user identities need to be challenged as Internet advances to reach all edges of human society. Through these projects, I show how large problems can be practically addressed through a combination of design, policy, and algorithms.
Sambasivan is a researcher focused on technology design for social, economic and political benefits in the Global South. Her research spans the areas of HCI and ICTD, and has won several recognitions at top conferences. She has been a researcher at Google since 2012, where she has co-founded a group to conduct future facing research on under-represented topics, such as gender equity and new technology users. Her research has influenced several large-scale real-world projects for the next billion users, and has been directly translated to core libraries, metrics, and guidance for Android and Web developers at Build for Billions, design.google/nbu, and Google I/O talks. Nithya has a Ph.D. and MS in information and computer sciences from University of California, Irvine and an MS in Human Computer Interaction (HCI) from Georgia Tech. Her dissertation focussed on technology design for the low-income communities of slums, urban sex workers and microenterprises in India. She is a recipient of Google’s Anita Borg and UC Irvine Dean’s fellowships. She has interned at Microsoft Research India, Nokia Research Center and IBM TJ Watson.
HCII Seminar: Thursday, April 5 at 1pm in GHC 6115
The idea is to use the mirror illusion to create a 3D animation. This is a perspective mapping project that uses a unity plugin to create 3D objects by modifying the models. This is inspired by videos on Youtube about creating holograms through the phone. It uses a coin to calibrate the location and additionally uses environment mapping techniques to create the optical map. The map is then invested to create the model through the unity code. They propose that this system can be used to create 3D animations such as virtual agents.
Move an object based on accelerometer movement. I used an ADXL362 and the wire protocol to move an object on the screen. The accelerometer sends the raw XYZ values and a state of a button, the P5Js scripts map the raw acceleration values to areas on the screen.
Todo: Need to update the p5js script and improve documentation and add photos.
The circuit shown above is a 3 pin led connected to a joystick. It uses the JoyStick movement to change the RGB values and the button press in the joystick is used to switch between states. The system has 5 states
Red constant, change G and B by moving the joystick
Green constant, change R and G by moving the joystick
Blue constant, change others through joystick
Reflection: if I were to redo this, I would transfer the states from 2, 3 and 4 making it a real color picker, and I also think the joystick button may not be ideal for an interrupt.