Intro: A close friend of mine has mostly unobtrusive Duane syndrome, meaning their right eye cannot turn outward, causing double vision, etc. whenever they look to the right. They always like and relate to chameleons or owls because of this. That relation got me thinking about how this syndrome could be experienced by others to help empathize. I brainstormed a game with their approval that is more or less a first person I, Spy starring a chameleon. Each of the players’ first person “eyes” are controlled independently, and the goal is to focus on certain objects you find. This would be controlled by two 3-axis trackballs, matching each eyes pitch, roll, and yaw. For this assignment, I prototyped this trackball, and in five years time, can see this alternate input method making a truly unique mark on a game.
Proof of Concept
The Trackball
Normal mouse trackballs often use a single optical sensor to read one plane of data, or, the XY coordinates of a screen. Optical sensors typically do this by comparing pixels between two images taken at a high frame rate. I figured it would be easy to harvest two optical sensors from two mice to get the XY data from one and rotate the other 90 degrees to get only the Z from it. This pair of a mouse and one orthogonal to it would give me all 3 axes.
I was wrong in how difficult it would be to find mice with documented pinouts, since I am not good enough yet to deduce them on my own. After going through seven or eight or so different mice, nearly all with different chips, I ordered two PS/2 mice. I would not access these optical sensors directly, but knew I could communicate with the PS/2 port as a backup. So, the project now had two mice talking to my Arduino effectively.
The ball would need to be very close to the mice, since they’re quite old (being PS/2 and all) and not that great at detecting motion. I used ball transfers and PVC pipe to allow the ball to rest over a mouse at its bottom with 3 points of contact, and a small elevated platform for the orthogonal mouse to get its third axis.
After going through a number of balls, a soccer ball was large enough to use the complete mice. I attempted disassembly of the PS/2 mice, but was too scared of breaking them and missing the deadline by having to order more, so kept them whole, leading to a foam molding housing.
The Game
I used a simple program Udino to talk to Unity, a game engine. Unity could read any port on the board, and use the data accordingly. To render first person double vision, I wrote a screen space shader that blended the views of two different cameras a few units apart in the world. The closer their forward vectors, the closer the cameras would also move to each other, effectively rendering the same “view” and making blending seamless. The less similar their forward Vectors, the more it would blend, and cause dissociation. Udino successfully controlled each camera separately. A view of the cameras working can be found here:
Unfortunately, I could not connect the working trackball to Unity to successfuly modify game state with the soccer ball. I am guessing it has to do with Udino using serial communication, and the PS/2 library I am using also needing to write calls to the mice over the serial port. In the future, I hope to get proper optical sensors instead of whole mice to make this project more compact and use less unnecessary protocols.
Conclusion
Overall, I am disappointed with failing to connect the trackball to Unity, but satisfied that all of the individual parts of the pipeline still work separately. Based on playtesting, the ball itself is a fun toy and easy to let people play with, which I think is important because it sets a base level of engagement. To even try to understand someone else’s experience, its important to want to in the first place, and I feel like this project is a good step in the right direction of this.
