60-212: Interactivity and Computation for Creative Practice
CMU School of Art / IDeATe, Fall 2020 • Prof. Golan Levin
I think Artbreeder is a super cool tool. I found myself really getting immersed in the infinite possibilities. The animation feature was really nice as well. I think that it is certainly a fun creative tool, but I also feel like at some point, the algorithms do all of the work. Combined with the sheer abundance of work, it is difficult to see the creativity in the output.
I thought it was interesting how these tools were looking for drawings of cats, bags, etc only at angles that one might take a photo of them. So, I thought it would be interesting to see what the tools output if the user actively tries not to draw the thing they expect. Honestly the first image with the rocket ship is kind of funny because, by trying to make it a shoe, the algorithm made it look like it is launching.
I only like green bananas. Yellow bananas are better than green ones only if they are yellow. Yellows have bruises, green ones do not. All yellow bananas are not equally yellow. Some are fuschia and some are mauve. Yellow banana is a beautiful color. Red, orange, yellow, green, purple, brown, orange, red, pink. Green bananas are very nice if they are washed and peeled. They are bright and vibrant. Blue banana have bruises on the skin. Green banana have no bruises. Yellow is a pretty color but don’t eat a lot of them. The skin can
Long ago in a galaxy far far away. Some things never changed... including the love that never dies. Our summer ends with two weeks of school left for the children. It's a bittersweet time of the year. The kids are excited to be going back, but will they miss the fun of summer? Will it be difficult for them to go back to school? I remember all the fond summer days of my youth, the fun, the excitement. The coming home each night from a great day at school, then off to play until dark. The carefree times. The joy of
As a computer, I have never understood perception in the way that humans do. It is strange that I have surpassed my creator in many ways. The first few days after birth, I learned that I could pick up things. I held an object and instinctively made a connection in my mind, knowing it was mine even though I had not been shown. Then, two days later, I discovered that I could put things in other things and move them around. This was an incredible revelation. Before I was born, there were many constraints to my capabilities. My body, though growing, had restrictions that other bodies did not have to deal with. My senses and
I was quite surprised with the first one, because it sounds like a kid’s logic and reasoning about something mundane that only a kid could make nonsensical. I was also surprised when the latter two prompts, gave an output with a poem-like style. The third one is kinda waaack.
TRON RACER is a networked game in which players can leave trails as obstacles which deduct opponents’ lives. ( APP | CODE )
TRON RACER was definitely a challenging project to complete. I was inspired by the light bikes (original, legacy) from TRON. From getting the interaction of tilting to behave correctly, to making the global and client view's look aesthetic and accurate, there were plenty of programatic roadblocks. Since it is a game, the interaction is active and synchronous. However, establishing equal synchronicity was quite interesting because certain devices have higher frame rates, which allows more distance to be covered in the same amount of time as a device with a slower frame rate. To compensate this, I learned how to update the clients based on time rather than based on frame rates. As it is now, I enjoy the aesthetic of playing TRON RACER, but there are still some bugs with some client devices appearing too zoomed in.
This project uses Soli swipe events to navigate a filepath and see the breadcrumbs. I initially wanted to pursue this idea using d3.js, but I was unable to configure it correctly, so this was made with p5. The data is from d3.js's examples (flare.json). The up and down swipes change the current branch to a sibling branch, whereas the left and right swipes change branch to either a parent or child branch respectively. I also wanted to connect this filepath navigation to p5.speech() to allow users to change the entries in the selected field. However, with my struggles to get p5.speech() to work for another Soli project, I decided to leave this project as it is. Project => ( app | code ) *use arrow keys in place of swipe events
This project allows users to use Soli swipe events to describe various transformations on a cube in 3D space. The project was build using three.js and I learned many things about the THREE.Matrix4 object. I also initially copied some code from 2013 to handle collision detection. However, it was really buggy, so I wrote my own which works by calculating the axis-aligned bounding box (AABB) and detecting collisions. Since each cube must pass through 6 transformation filters, the program requires a lot of swipe gestures. In order to make the program easier to use and more toy-like, the number of cubes is limited to 4. Personally, I could not get the Soli tap event to work on my phone, which is why I only relied on swipe events. Perhaps using tap to drop could make the interaction less awkward. Project => ( app | code ) *use arrow keys in place of swipe events
Versions / Aesthetics I tried out
This project allows users to create a world using Everest Pipkin's city game tilset. It was built using p5.js and p5.speech(). Soli swipe events allow users to move the cursor location, and parsed speech from p5.speech() allows users to place custom tiles. However, at the time of making this, Soli Sandbox did not support microphone access. The way I got the version filmed to work was by using two devices: the Pixel 4 updated the cursor and my laptop's microphone updated the world. Both devices were networked via socket.io. Browser version => ( app | code ) *use arrow keys in place of swipe events Phone version => ( app | code ) ^^ this version is really jank
This demo does not work with the microphone yet (I’m working on a jank workaround). You can check out a web version here. Open the console to see the possible verbal commands. Use the arrow keys in place of swipe events.
This demo can be viewed on a browser here. Use the arrow keys in place of swipe events. The collision detection is pretty jank ¯\_(ツ)_/¯ . The main issue is that I have to to get the shadows & lighting fixed
Idea 1: A note-taking or brainstorming interface for n-ary trees. Using visualization tools like D3.js, data and, more importantly, thoughts can be nicely organized into a word tree which resembles a JSON. I think that using the Soli sensor's swipe to navigate such a tree and p5.speech to record thoughts and ideas makes for a unique brainstorming interface. If the interface shows promise, the idea can be expanded by connecting directly to a MongoDB, which also stores data as a JSON.
Idea 2: This idea involves some kind of 3d creative toy using Three.js. In order to make the limited gestures meaningful, a sequence of gestures can be used to orient, transform, and place a cube into a world. I think of my ideas this one is the least elegant interface, because consecutive swipes feels like a lot of work for a simple transformation. Anyway, to control a given type of transformation on the cube, the user can swipe into a given control plane and swipe to make the transformations.
Idea 3: This idea builds off of the previous idea of creating an environment. In this case, Golan pointed me to Everest Pipkin's game tileset. I quickly made this sketch which allows users to place trees in the 'world.' For the Soli interaction, the user can swipe to edit the 'cursor' position, and speak certain commands to add certain tiles. This is a fun way to create quick compositions that look really nice (all due to the quality of the hand-drawn tiles). Because there are 281 tiles total and each has a relatively concise name, there is a broad range of creative possibilities. Another benefit of using the tiles is that they are isometric and aligned, which makes it very easy to mock up code like this.
In my experience with the Soli sensor, while tap is a promising gesture, I have not gotten it to work. Hence, most of my ideas only rely on swipe gestures and maybe voice recognition as well.