Category: 05-Pix2Pix

Cats are always fun. Not many surprises here as it’s just a classic Pix2Pix model. But I was surprised to find that many art projects, including this one (edges to cats) and the “Learning to see: Gloomy Sunday” we saw in class were actually included in the original Pix2Pix paper, which I thought was kinda cool. This site seems to run inference on the frontend and is quite unstable (I would get “page unresponsive” multiple times for each generation.)

Koke_Cacao-Pix2Pix

(05-Pix2Pix)

I have played this model years ago. The model tries very hard to fit both low-level pattern and border structure to
the input data, which creates funny-looking images. But the model is only good at interpreting thin lines, with
equal stroke weight and without color.

Above is two examples I made with edges2shoes. I notice that it works best it you draw realistically with perspective(image 1). Whereas a highly stylish, 2D drawing(image 2) doesn’t render as well.

It was pretty interesting seeing how the algorithm would interpret my edges, since it was unpredictable, and I liked seeing how a doodle could be easily transformed into a fleshed out image.

I tried to put windows in the background, but the background turned into a wall.

I spent a few runs learning the “rules” of the kinds of forms and details the model responded to (narrow paths for zippers, loops like in strappy sandals, wavy bottoms for soles…) then tried to figure out how to push them. For facades I followed the rules for the most part, here trying to see if I could believably render a facade upside down. For shoes, I played more with how the language of “shoe-like” contours can be composed in unexpected ways. It seems the more interesting results come from composing familiar details in new ways, as the model responds well to details and composes the image however the input dictates – and this gets at a balance of “crystallization” (human-recognizable details) and “chaos” (unexpected compositions).

This tool allowed me to eliminate my preceding knowledge of cats and buildings while making these drawings. I felt that this tool gets me to be more creative with the final output. It was interesting to be able to observe the output changing by minor changes that I make, such as adding a new rectangle.

Since I have already played with the edges2cats software, I decided to play around with myHeritage instead.

This first video is of my grandfather who just passed away last week. I thought of using him because I was maybe considering sending it to my parents; however, even though the intentions of this software are good, it can also be seen as creepy and lacking awareness, which is something that needs to be overcome with this type of technology as a whole.

I am including this second video of my still living grandma because it shows how the technology is not yet perfect(even though we cannot expect it to be perfect when the only data it has is from a flat picture and not a 360 view). Her face is warped slightly.

I have never seen more cursed cats in my life than this and I love it. I wonder if it’s possible to create a normal looking cat if the drawing is good enough.