We did more literature research with the goal of focusing on a single sensing technique, and we realized that work in this space is much more advanced and complex compared with the approaches that we were looking at before. The two possible technical directions that we eventually narrowed down to are touch tracking with electric field tomography and capacitive sensing.
Electrick uses electric field tomography in concert with an electrically conductive material to detect touch inputs on large surfaces that does not depend on distribution of the electrodes. Its following work Pulp Nonfiction further explores using Electrick with paper interfaces to track finger and writing instruments inputs.
We took inspiration from this and wondered if we could use the same system for fabric and explore where to place conductive yarn, and the density of conductive yarn. One immediate difficulty we see with this approach is that Electrick uses a custom sensor board and its own sensing system, so we are concerned with spending a long time reinventing wheels and further if we are even able to reinvent the wheels.
Another inspiration we got was the I/O braid. It uses conductive and passive yarns to create a helical sensing matrix. The I/O braid is able to detect many gestures such as twisting, flicking, sliding, and etc. based on capacitive signal strengths. We wanted to explore how we can apply the same technique in the 2D space. The paper mentions using a PSoC® 4 S-Series Pioneer Kit, but we weren’t able to find much information about its difference compared with other capacitive sensor chips.
For both approaches, there is much to explore to achieve the same goal of detecting gestures(even as simple as being able to detect where the textile is being touched, i.e. a textile touchscreen) on a textile surface. This adds to our kids story book narrative as it allows a wider interaction space and thus contributes to a likely open-ended book, since we could combine this position and touch sensing technique with the interaction of placing assets on to or just directly touching different locations of the page.
For the first approach, we can’t really think of a clear proof-of-concept experiment as of now because of our lack of technical understanding in electric field tomography.
For the second approach, the proof of concept experiment could be hooding up a capacitive sensing board with several different sensing ‘matrices’ using conductive copper tape(thicker and easier to work with than conductive yarn for now).
Our aim for the outcome is to focus on one method of sensing touch input and extend the interaction model from the basic “touch trigger outputs” to capturing different types of touch inputs(like the microinteractions from the second paper) or locationized inputs.
We’d love to have some feedback and advice, especially on the feasibility of this direction and anticipated technical challenge, and more related technologies/references.
References:
Yang Zhang, Gierad Laput, and Chris Harrison. 2017. Electrick: Low-Cost Touch Sensing Using Electric Field Tomography. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI ’17). Association for Computing Machinery, New York, NY, USA, 1–14. DOI:https://doi-org.proxy.library.cmu.edu/10.1145/3025453.3025842
Yang Zhang and Chris Harrison. 2018. Pulp Nonfiction: Low-Cost Touch Tracking for Paper. Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. Association for Computing Machinery, New York, NY, USA, Paper 117, 1–11. DOI:https://doi.org/10.1145/3173574.3173691
Alex Olwal, Jon Moeller, Greg Priest-Dorman, Thad Starner, and Ben Carroll. 2018. I/O Braid: Scalable Touch-Sensitive Lighted Cords Using Spiraling, Repeating Sensing Textiles and Fiber Optics. In Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology (UIST ’18). Association for Computing Machinery, New York, NY, USA, 485–497. DOI:https://doi-org.proxy.library.cmu.edu/10.1145/3242587.3242638
Alex Olwal, Thad Starner, and Gowa Mainini. 2020. E-Textile Microinteractions: Augmenting Twist with Flick, Slide and Grasp Gestures for Soft Electronics. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (CHI ’20). Association for Computing Machinery, New York, NY, USA, 1–13. DOI:https://doi-org.proxy.library.cmu.edu/10.1145/3313831.3376236