Author: ctriplet@andrew.cmu.edu

I'm Conor and I'm a 2nd-year grad student at the Entertainment Technology Center. I got my undergrad in Mechanical Engineering (with minors in Design and Psychology) from the University of Notre Dame. I am really interested in theme park design, experience design and integrating physical components into XR experiences. I have some Arduino programming experience (it's been a few years), have designed different experiences for ADA compliance, and have full use of the ETC woodshop.

Final Crit: Accessible Theme Parks

Final Project: Theme Park Wearables

Target Audience: Children and adults on the Autism Spectrum attending theme parks

Background: A recent trend in the theme park industry has been making experiences and attractions more accessible to guests of all backgrounds and abilities. Whether it be physical and mental considerations, theme parks are making efforts to give all guests the chance to experience all of the emotions they hope to evoke within the park. Some rides and experiences have gone so far to be retrofitted to allow wheelchairs onboard or guests can get closed caption devices throughout the park so they can pick up various cues that they may not be able to hear. Lots of the current attempts to make parks more accessible, though, only focus on the rides and experiences themselves; very few solutions approach the time spent not on rides… which makes up almost 90% of the guest’s day. One specific demographic that struggles with the non-ride time are those on the autistic spectrum. There are a few major reasons for these struggles, two of which I will start to focus on. Firstly, waiting is very difficult. One of the documented struggles those with autism face is the keeping of time (https://www.ncbi.nlm.nih.gov/pubmed/25078724, https://www.spectrumnews.org/opinion/for-people-with-autism-time-is-slippery-concept/). Time management and understanding passage of time calls on various parts of the brain depending on short-term vs. long-term memory formation; however, those with ASD call upon both parts of the brain at all times, causing a mix-up in a person’s perception of time. Considering you spend most of your time at a theme park waiting in line, if it is hard to keep an accurate track of that throughout the day, it can be very frustrating and exhausting which are not the feelings that should be evoked while waiting for a roller coaster. In conjunction with that, since theme park lines and long and theme park designers know this, they go to extreme lengths to keep people entertained while they are in line. Whether with mobile games or physical activities and interactives, more and more line queues are trying to be interactive to keep people happy in line. While a great idea, a lot of these interactive installations can provide sensory overload to guests (https://iancommunity.org/challenging-behaviors). They are often loud, filled with flashing lights and have elements of surprise within them. Coping with sensory overload is another well-documented struggle of those with ASD and combining that with already-present waiting frustration can trigger even more intense negative reactions.

General Questions: 

  1. Is there a way to communicate wait times while guests are in line that is non-invasive and does not rely on numbers or comparisons to other wait times? 
  2. Are there ways to help guests combat adverse reactions to overstimulation at theme parks?

General Solutions: 

  1. Many theme parks already have a wearables infrastructure that most, if not all, guests use that act as their park ticket, credit card and way to interact with more things in the park (https://disneyworld.disney.go.com/plan/my-disney-experience/bands-cards/). Utilizing that wearable would be fairly noninvasive in terms of not adding cumbersome pieces of technology to a guest’s day or giving them another app for their phone. A lot of the literature when it comes to teaching children with ASD to manage time efficiently focuses on consistent scheduling and visualizing that scheduling. While theme park plans are known to vary wildly and can change at any moment, they do not change very much once you reach a ride. Giving a guest a consistent visual reminder of the wait time and their place within the line would provide a nice benchmark for guests to look at every time they get in line. In addition, it could be a fairly simple physical representation, like LEDS within the wearable, of where you physically are in the line queue as opposed to how much time is left. 
  2. Theme parks are synonymous with being “extra” and over-stimulating and I do not see them slowing down any time soon. Some theme parks have added “sensory waiting rooms” for guests to go and slow themselves down, calm their sense and the like in order to avoid some of the sensory overload (https://www.familyvacationcritic.com/best-autism-friendly-theme-parks/art/). I struggle with these rooms because then you are taking guests out of the experience that they want to have/are paying for. I believe you could take the same wearable infrastructure and embed physical feedback within it to allow guests to “cope” with the stimuli when it gets to be too much. The wearable could be outfitted with various sensors to track external stimuli (as well as a heart rate monitor or similar to track a guest’s response to stimuli) and once the external stimuli crosses a certain threshold, then the physical feedback kicks in to help center the guest by triggering certain breathing patterns. Research shows that when children with ASD face too much stimulation, they begin to “emotionally breathe” – essentially fast, large breaths where their chest cavities contract and expand rapidly (https://breathing.com/blogs/autism/optimal-breathing-autism-brain-development). This type of breathing is hard on your lungs and it cuts off blood-flow to the brain, both making it very difficult to stop and come back from. Using timed feedback, like haptic buzzes or similar, to guide guests to breathe at a consistent, safe, and calming clip could help guests to overcome some of the sensory overload and potentially still get to enjoy some of the experiences that they could not in the past (https://hal.inria.fr/hal-01609385/document). 

Proof of Concept: 

Done in two parts:

  1. Line Queue Model: used to illustrate guest spot in line based on RFID checkpoints and visualized by “bar graph” LEDs
    1. Cardboard model outfitted with RFID cards underneath the line at various spots, signaling different checkpoints in the line… in 5 years/with a theme park budget, these checkpoints could be incorporated more seamlessly into fence posts or other props within the line queue with better sensors that do not have a 3cm read distance
    2. RFID reader represents human walking through the line
    3. Bar graph LEDs visualize the progress through the line on the guest’s wrist
  2. Sound-Driven Motors: surprise moment that triggers a stimulating event that crosses a sensory overload threshold
    1. Sound sensor used to take in external data/factors within each bracelet
    2. Tactor/vibration motor used to create haptic feedback pattern to guide guest in breathe in and out in rhythm to allow them to calm down within the environment

Potential Down the Road:

With the success of RFID-driven wearables and interactive props and items, theme parks will likely begin exploring more ways to use them and potentially use them for more than just driving revenue. As sensors get smaller and smaller, more of them could be added to the bands to provide different levels of feedback both to and from the guest. If an attraction calls for guests to be cold, a smaller and less dangerous Peltier plate could evoke that feeling OR if a guest’s heart-rate gets to be too high in a simulator or other experience, their family members of maybe even staff can be notified if it gets into a dangerous territory. The slippery slope then presents itself of “how much data/sensor activity is too much?” and we can argue about that as the day is long, but there is the opportunity for more technology to be implemented in these devices. the potential addition of AI-driven “tracking/status monitoring” could also prove to be beneficial, having machines learn trends in line queues and wait times based on previous day/week/month riders or ride break-down similar or the like. At any rate, these pieces of technology are here to stay and people will expect more and more out of them during these fantastical experiences, they should be able to do more for everyone. 

Final Crit

Sound Crit: Airplane Headphones

Problem: Headphones on Planes

Description: Does anyone not wear headphones on a plane anymore? Whether you are listening to an audio book or a movie or white noise and trying to fall asleep, people are constantly connected to something in the air and that can make lots of jobs difficult. 

  • Seat mate: if you want to get out, but do not want to tap the person next to you to interrupt them, what do you do?
  • Flight attendant: do you need to ask everyone to take their headphones out every time you walk by for drink orders, pretzels or trash?
  • Captain: should you even give announcements if most people are not listening in the first place?
  • Flyer: when do I turn my music down to get information?

Since I do not see people returning to headphone-less days on planes in the next five years, I think our headphones need to be smarter.

General Solution: Headphones that adjust their own volume based on given situations.

Proof of Concept: Using a Unity build to go through multiple situations, I have an interactive flight attendant and another person on the flight that both move up and down the rows, causing different levels of audio feedback. The other flyer does not change the volume as they are likely just going to the bathroom or stretching, but the flight attendant gradually lowers the volume as they approach just in case they need to talk to you. ALSO, depending on their speed, that gradient could change because the faster they are walking then they are probably just going to the other end of the plane and not stopping en route. As the attendant stops at your chair, if you look at them, your music’s volume completely cuts out and returns when you look away. Finally, represented by a key-click, if you are asleep and want to stay asleep because of the flight attendants, your music will not change at all as they walk through. In this mode, the only thing that will change the volume of your music is a captain’s announcement because of its safety implications. In real life and not a computer simulation, each flight attendant would be outfitted with some kind of RFID tag/sensor that communicates with a corresponding sensor in each seat. Since your phone is connected to your seat number because of your reservation, they would sync to provide you with accurate location data. 

Unity files: https://drive.google.com/open?id=1OzPjUNTmd8tUH93fNJEz809hihKvDIr7

Assignment 8: Fan-Powered Door Chimes

State Machine: Distance Door Bells

Problem: Door bells are usually intrusive noises. They are sudden and loud and usually disturb the peace, or at least peace hoped for, in your home. In addition, most door bells only have two states: at the door or not. Because of this, guests usually must wait at the door for the host to arrive at the door and the host usually must drop everything they’re doing immediately to go and get the door. Is there a way to create a simple doorbell that does not alarm everyone in the house and also gives you a sense of where people are relative to the door?

General solution: A door chime system that created more calming/less disturbing sounds in order to alert the host. The system can also relay more information, namely, how far away someone is from the door to symbolize how much time the host has to finish whatever they are doing before making their guest wait at the door. As a guest gets closer and closer to the door, the chimes will grow louder and louder. Also, doorbells are traditionally known, at least at my house, for alarming dogs and setting them into a fit. To avoid this, these chimes can also be turned to a “do not disturb” mode for when people aren’t home or are asleep, so they chimes do not make noise to disturb the peace.

Proof of Concept: My system utilized the following components:

  • Ultrasonic Sensor: used to measure distance to simulate guest proximity to door
  • Fan: the driver of the chimes, causing them to knock together and create noise depending on the fan speed determined by the distance sensor
    • Transistor: acts as a voltage gate to allow the fan to be controlled at various speeds
    • Fan has 4 different states of sound production: just fan noise with no air movement, light air with light chimes, medium air with medium chimes, heavy air with heavy chimes
  • Push button: acts as an interrupt that changes the “mode” of the doorbell

Assignment8_FanDoorChimes

Files above, but after potentially shorting Chance’s computers and digging deeper into my own arduino issues… I am going to skip the video portion to get those things figured out for now.

Assignment 7: Kitchen Bells

State Machine: Ovens

Problem: While cooking, a chef must keep an eye on two different temperatures in order to make sure what put in an oven is cooked through. The oven temperature is easy enough, most ovens print their temperature on an LCD screen and some even “beep” when it reaches a target temperature. That’s great, but the more important number, one could argue, is the internal temperature of whatever is in the oven. How can someone keep track of that temperature without opening the oven to check/disturbing the oven temperature?

General solution: A (5 year from now quality) temperature sensor that transmits the food’s internal temperature and translates its “done-ness” not through numbers, but through sound frequency. Sound would potentially allow for a better gradient for understanding where your food is at in its cooking progression and it would also allow chefs to know the status at all times even if they leave the room or start looking at other things in the kitchen. 

Proof of Concept: My system utilized the following components:

  • A temperature sensor: used to read ambient temperature in the demo, but symbolizes the internal temperature of a food
  • Servo: acts as output mechanism to create sound patterns to relay information about food’s cooking state
    • Right now, I have the delays set up based on different ranges between the “final temperature” of your food and its “current temperature”. For the servo, it makes the most sense because it would be pretty difficult to distinguish between single digits on angles of rotation (frequency) as opposed to how often it oscillates between two points. If I would have used a speaker, I probably would have set it up so that the frequency/pitch of the sound generated by the temperature difference acts as a gradient because it is relatively easier to tell if a sound is getting higher or lower-pitched (even for the musically-challenged like myself). 
  • Push button: acts as an interrupt that signifies opening the oven, thus alerting the user that something is wrong (like they left the oven open or someone else checked)

Further Development: With the servo, if I would have had fewer things to work on this weekend, you could also turn this system into a set of chimes so that you aren’t working with mechanical sounds as an output. You could string pieces of metal or glass or other things around the servo and they could hit each other to produce more delicate/harsh/whatever-kind-of-sound-you-want-for-your-home sounds.

Assignment7

Alarm Bed 2.0

Problem: Waking up is hard. Lights don’t work. Alarms don’t work. Being yelled at doesn’t work. You have to be moved to be woken up. But what if no one is there to shake you awake?

Target Audience: Deaf/hearing-impaired children. Proper sleep is a habit that needs to be taught from a young age to help lead to healthier lifestyles later in life. Children with hearing impairments face another barrier to “learning how and when” to sleep because they miss some of the important audio cues that trigger and aid sleep. First, there is a high level of sleep insecurity for children with hearing impairment. Children that can hear are usually comforted by their parents’ voices during a bedtime story or by the normal sounds they hear around the house; however, children that cannot hear do not have that luxury – they have to face total silence and darkness alone, which is a scary thing for people of all ages. In addition, some of these children use hearing aids during the day which gives them the even sharper/more noticeably disturbing reality of some noise throughout the day and then total silence.

General solution: A vibrating bed that takes in various sources of available data and inputs to get you out of bed. Everyone has different sleep habits and different life demands, so depending on why you are being woken up, the bed will shake in a certain way. How?

  • Continuous data streams
    • Google Calendar: if an accurate calendar is kept by a child’s parents and they can program certain morning routines like cleaning up and eating breakfast, your bed could learn when it should wake you up for work/school
      • Can make this decision depending on traffic patterns/weather/other peoples’ events (kids, friends, etc.)
      • This process could also teach kids how to plan their mornings and establish a routine.
    • Sleep data: lots of research has been done on sleep cycles and various pieces of technology can track biological data like heart rate and REM stage, your bed could learn your particular patterns over time and wake you up at a time that is optimal within your sleep cycle
  • Situational
    • High Frequency noises: if a fire alarm or security alarm goes off, a child that cannot hear would usually be forced to wait for their parents to grab them and go. This feature could wake them up sooner and help expedite a potential evacuation process
    • “Kitchen wake-up button”: Kids do not always follow directions… so here, a parent can tap a button in a different room to shake the bed without having to go into their child’s room
      • Button system also has a status LED that shows
        • Off = not in bed
        • On = in bed
        • Flashing = in bed, but alarm activated
  • Interacting with User
    • Snooze: if sleeper hits a button next to their bed three times in a row, then the alarm will turn off and not turn back on
    • Insecure Sleep Aid: if the bed senses that a child is tossing and turning for a certain amount of time as they get into bed, then it can lightly rumble to simulate rubbing a child’s back or “physical white noise feedback”
    • Parents: if your kid gets out of bed, you can have your bed shake as well if this is linked throughout the house

Proof of Concept: I connected the following pieces of hardware to create this demo:

  • Transducers: shakes bed at a given frequency
    • One located by the sleeper’s head
    • One located by the sleeper’s feet
  • Potentiometer: represents an audio source
    • The transduer turns on to different intensities depending on where the potentiometer is set to
  • Push button 1: represents the “kitchen wake-up button”, works as an interrupt within the program
  • LED : a part of “kitchen wake-up button” that represents status of bed
  • Push button 2: represents the “snooze” feature of the bed, where a sleeper can turn off the rumbling to go back to sleep
  • Flex Resistor: represents a sensor in the bed that determines if someone is in the bed or not

Summary: A device like this could help children (and adults) who cannot hear to feel more secure in their sleep and encourage healthier sleep patterns. One of the biggest potential challenges with the device would be finding a powerful enough motor/transducer that can produce a variety of vibrations across a heavy bed/frame (and the quieter the better, for the rest of the home’s sake). Even with that challenge, though, everyone should have a good night’s sleep and this could be a way to provide that.

Files: KineticsCrit

Alarm Bed

State Machine: Sleeping

Problem: Waking up is hard. Lights don’t work. Alarms don’t work. Being yelled at doesn’t work. You have to be moved to be woken up. But what if no one is there to shake you awake?

General solution: A vibrating bed that takes in various sources of available data and inputs to get you out of bed. Everyone has different sleep habits and different life demands, so depending on why you are being woken up, the bed will shake in a certain way. How?

  • Potential continuous data streams
    • Google Calendar: if an accurate calendar is kept and you can program certain morning routines like cleaning up and eating breakfast, your bed could learn over time when it should wake you up for work/school depending on traffic patterns/weather/other peoples’ events (kids, friends, etc.)
    • Sleep data: lots of research has been done on sleep cycles and various pieces of technology can track biological data like heart rate and REM stage, your bed could learn your particular patterns over time and wake you up at a time that is optimal within your sleep cycle
  • Situational data streams
    • High Frequency noises: if a baby cries in the room next door or one of your home’s alarms goes off, your bed could shake in a faster/more violent manner to make sure to get your attention
    • “Kitchen wake-up button”: if one of your roommates or family members won’t get out of bed, you can flip a switch in a different room to shake the bed without having to go into their room

Proof of Concept: I connected the following pieces of hardware to create this demo:

  • Servo motor: represents the shaking bed
  • Potentiometer: represents a timer, as well as higher frequency sounds (main sources of input/data)
    • The motor turns on to different intensities/patterns depending on where the potentiometer is set to
  • Push button: represents the “kitchen wake-up button”, works as an interrupt within the program
  • Slide switch: represents the “off button” for the bed, works as an interrupt within the program

Assignment 6

Interactive Theater

Chance and I went to Bricolage Production Company’s latest creation, Project Amelia, tonight. It’s “a next-level immersive theater experience that invites you to the R&D lab of Aura, one of the world’s most innovative tech giants, to participate in the launch of a groundbreaking intelligence product like no other.” Their words, not mine. It’s a cool take on traditional theater with lots of clever uses of simple interactive devices that we could make for this class. You wear an RFID tag bracelet throughout the show to interact with various games and demos that are aimed at teaching guests about the power of artificial intelligence. The show runs for a few more weeks, so check it out!

Project Amelia

Assignment 5: Pump Your Breaks

Data Set: Highway Police Locations

2024 Assumptions

  • People are still driving because autonomous vehicle timelines have been pushed back (again)
  • Police departments have become more transparent to assure citizens of intentions (or more people submit data on police locations to Waze)

General Problem: One of the most common sources of frustration and/or stress during road trips is running into a speed trap on the highway. No matter what speed you are going, everyone seems to tense up for split second when they see a police car in the distance. Sudden slow-downs or stops on the highway can be extremely dangerous, especially when other drivers do not slow down accordingly. Is there a way to alert drivers as to law enforcement locations to give them ample time to adjust speed safely?

General Solution: Utilizing different sources of data, cars could install haptic feedback systems in steering wheels to alert drivers as to law enforcement locations. When you see a police car, it is often too late to really change your speed, especially safely. This haptic feedback system would take in data sourced from either local police departments (maybe) or crowd-sourced from apps like Waze to determine where police cars are lined up. Based on their locations, your car’s location and speed, the current traffic and weather circumstances and potentially geography of the area, this new gadget would determine a certain distance from the police car that you should begin to adjust your speed in order to avoid getting a ticket and doing so safely. The best feedback for this gadget would be two points of haptic feedback in the wheel. This would allow the driver to still focus on the standard audio and visual pieces of feedback their car already gives them, but a strong buzz on the wheel would be hard to ignore.

Proof of Concept: This prototype uses the following materials:

  1. 2 Vibrating Motor Discs: to represent the haptic feedback; one for the top of the wheel to alert driver to police ahead, one for the bottom of the wheel to alert the driver to police behind (see image below)
  2. 2 Ultrasonic sensors: to represent car’s distance from police car (one behind, one ahead)

Video coming tomorrow (Tuesday) (fickle vibration sensors – want to lay them down/tape them down in classroom)

More Thoughts: I wanted to tackle the vibration motors because Jet mentioned it was hard to smooth them/make them less noticeable or shocking to people. I tried a few things and nothing quite did the trick which is why I ended up “coding” the varying distances using patterns not intensity. Also, these vibration sensors are tiny and fickle little pieces of hardware for future reference.

A5_PoliceScanner

Crit #1: Honest Visualization of Intoxication

State Machine: Bar Patrons

Problem: In undergrad, my friend sent me a picture of myself from a night out the week before. I was in the middle of a crowded bar, looking for what could have been a beer, the people I came with, or my own sanity. The caption could have been “an island that cannot hear in an ocean that cannot see.” 

Bars are perfect examples of places where design needs to realize that IQs and senses diminish rapidly, in addition to inhibitions. While people usually go to bars in groups, those groups quickly dissipate as people go the dance floor, the bathroom, to find other friends present and more. Combine that with usually spotty reception and barely enough room to operate your phone in the sea of bodies, communication is tough; communication about the health and safety of your friends is even tougher. Even if by some miracle you find your friends to check on them, can you really believe them if they’ve said they’ve only done one shot, but look like they are ready to fall over? Your sense of perception is off and the resulting communication is therefore unreliable. 

Is there a way to represent your and others’ state-of-being at a bar at a glance? Can that system or product be smart and take advantage of certain pieces of data to give the most accurate diagnosis?

General solution: A sensor and LED-outfitted bracelet that takes pieces of environmental and user-entered data to determine how intoxicated you or your accompanying friends are. Using a color-coded system, you can easily recognize who is doing well, who needs your help, and who is ready to go home without pulling out your phone or searching around.

Proof of Concept: My system utilized the following components:

  • 2 RGB LEDs (and resistors) – both on your bracelet; one to represent your intoxication level, another to represent your friend’s
  • 2 push buttons (and resistors) – one on your bracelet to allow you to signify another drink (+1 press), you want to go home (press until light is blue, you are in danger (spam press until white flashing); another to be used for a demo to signify your friend changing their state
  • 2 analog temperature sensors – one inside your bracelet to record your body temperature; one used for the demo to signify your friend’s
  • p5.Js – monitoring for those who do not have a bracelet

Following research from different sources, I wanted to apply various environmental factors that affect someone’s level of intoxication or their perception of it. One study cited body temperature as one such factor; if someone is drinking somewhere where it is very hot or very cold, alcohol typically masks your internal temperature such that you could be drinking yourself into sickness without knowing it. Because of that, I wrote the code in such a way that if your body temperature leaves a certain range, then your intoxication/danger levels rises faster than it would otherwise. Other studies show that a simple passage of time allows some people’s intoxication levels to drop naturally; therefore, there are caveats in my code that say if a certain amount of time has passed between drinks, your intoxication level actually goes down.

Challenges: Trusting people as they drink. The whole system is predicated on people being honest about when they have another drink, which I’ll admit, could potentially be a bit of a stretch to expect out of people. There are also lots of other factors that go into determining a person’s intoxication level – type of alcohol, food consumed, water consumed, weight, gender, etc. – that cannot be tracked with sensors in a bracelet.

Opportunities: With further development, this would probably be best served to be paired with an app interface that allows people to build their own profile to be linked to their bracelet. You could enter more of your own data (like weight, gender, location, etc.) that would affect the aforementioned “ranges” for things like body temperature. The app would also allow you to track your habits over time and, with the advent of more robust machine learning techniques, your app/bracelet could help you stay safer or smarter while you’re out by altering when your color codes change based on your tolerance or suggesting you call an Uber at a certain time in the night. Also, I attempted to add in a p5.js component that could eventually evolve into a more sophisticated app that allows friends back in the dorm or at home to know what is going on – whether you are on your way home, whether they should have a Gatorade ready for your return, or more depending on your state. 

Assignment5 Files (Fritzing, Arduino, p5.js)

CMU’s Own Designing for Accessibility

I was watching college football the other day and caught this commercial during one of the breaks in the action. At first, I was reminded of this class because of the content (designing for accessibility), but then I realized the woman in the commercial was walking around CMU and Pittsburgh. Chieko Asakawa is an IBM fellow and a CMU professor… and blind. She’s widely regarded for her work designing everyday objects for the visually-impaired.

She also has a TED Talk that is pretty widely cited for the benefits of “accessibility” design for the entire population.