{"id":1152,"date":"2022-02-23T01:20:58","date_gmt":"2022-02-23T06:20:58","guid":{"rendered":"https:\/\/courses.ideate.cmu.edu\/60-212\/s2022\/?page_id=1152"},"modified":"2022-02-28T08:30:17","modified_gmt":"2022-02-28T13:30:17","slug":"machine-learning-i","status":"publish","type":"page","link":"https:\/\/courses.ideate.cmu.edu\/60-212\/s2022\/daily-notes\/02-28-machine-learning\/machine-learning-i\/","title":{"rendered":"Machine Learning I"},"content":{"rendered":"

\u201cMore Like This, Please\u201d<\/h2>\n

Much of today\u2019s presentation focuses on the use of deep convolutional neural networks in generating imagery.<\/em><\/p>\n

Recall how earlier in the semester, we discussed computer arts pioneer, Vera Moln\u00e1r, born 1924 (and still working at 98), a Hungarian-French artist who was one of the first dozen or so people to make art with a computer. Moln\u00e1r described her interest in using the computer to create designs that could \u201csurprise\u201d her<\/em>.<\/p>\n

\"\"<\/p>\n

Consider this generative 1974 plotter artwork by Moln\u00e1r, below. How did she create this artwork? We might suppose there was something like a double-for-loop to create the main grid; another iterative loop to create the interior squares; and some randomness that determined whether or not to draw these interior squares, and if so, some additional randomness to govern the extent to which the positions of their vertices would be randomized. We suppose that there were some variables, specified by the artist, that controlled the amount of randomness, the dimensions of the grid, the various probabilities, etc.<\/p>\n

\"\"<\/p>\n

Molnar\u2019s process is a great illustration of how generative computer artworks were created during their first half-century. The artist writes a program that renders a form. This form is mathematically governed, or parameterized<\/i>, by\u00a0variables<\/b>\u00a0\u2014 specified by the artist. Change the values of these variables, and the form changes. She might link these variables to\u00a0randomness<\/b>, as Molnar does, or perhaps to gestural inputs, or perhaps link them to a stream of data, so that the form\u00a0visualizes<\/i>\u00a0that data. She has created an artwork (software) that makes artworks (prints). If she want \u201cMore like this, please\u201d, she just runs the software again.<\/p>\n

Just as with \u2018traditional\u2019 generative art (e.g. Vera Moln\u00e1r), artists using machine learning (ML) develop programs that generate an infinite variety of forms, and these forms are still characterized (or parameterized) by variables. What\u2019s interesting about the use of ML in the arts, is:<\/p>\n

    \n
  1. \n
      \n
    1. That the values of these variables are no longer specified by the artist. Instead, the variables are now deduced indirectly<\/em>\u00a0from the training data that the artist provides. As Kyle McDonald has pointed out:\u00a0machine learning is<\/em>\u00a0programming with examples, not instructions<\/em>.<\/strong>\u00a0<\/span>Give the computer lots of examples, and ML algorithms figure out the rules by which a computer can generate \u201cmore like this, please\u201d.<\/li>\n
    2. The use of ML typically means that the artists\u2019 new variables control\u00a0perceptually higher-order properties.<\/strong><\/em><\/span> (The parameter space, or number of possible variables, may also be significantly larger.) The artist\u2019s job becomes (in part) one of\u00a0selecting, curating, or creating training sets<\/em>.<\/li>\n<\/ol>\n<\/li>\n<\/ol>\n

      Here\u2019s\u00a0Helena Sarin\u2019s\u00a0Leaves<\/em><\/a>\u00a0(2018).<\/strong>\u00a0Sarin has trained a\u00a0Generative Adversarial Network<\/em>\u00a0on images of leaves: specifically, photos of a few thousand leaves that she collected in her back yard.<\/p>\n

      \"Leaves<\/p>\n

      GANs operate by establishing a battle between two different neural networks, a generator and a discriminator. As with feedback between counterfeiters and authorities, Sarin\u2019s\u00a0generator<\/em>\u00a0attempts to synthesize a leaf-like image; the\u00a0discriminator<\/em>\u00a0then attempts to determine whether or not it is a real image of a leaf. Using evaluative feedback from the discriminator, the generator improves its fakes\u2014eventually creating such good leaves that the discriminator can\u2019t tell real from fake.<\/p>\n

      Here is\u00a0Robbie Barrat\u2019s\u00a0Neural Network Balenciaga<\/em><\/a><\/strong>\u00a0(2018):<\/p>\n

      \"Balenciaga<\/p>\n

      Where Helena Sarin has been training networks using collections of leaves, Robbie Barrat has trained them with a different \u201cfall collection\u201d: photos of high couture from Balenciaga that he downloaded from the Internet. Barrat collected a large dataset of images from Balenciaga runway shows and catalogues, in order to generate outfits that are novel, but at the same time heavily inspired by Balenciaga\u2019s recent fashion lines. Barrat has pointed out that his network lacks any contextual awareness of the non-visual functions of clothing (e.g. why people carry bags, why people prefer symmetrical outfits) \u2013 and thus produces strange outfits that completely disregard these functions \u2014 such as a pair of pants with a wrap-around bag attached to the shin, and a multi-component asymmetrical coat with an enormous blue sleeve. Barrat\u2019s project circulated on the Internet; soon he was approached by a fashion company, which used his designs as inspiration for real artifacts.<\/p>\n

      For some artists, GANs present a way of understanding and simulating genres of human creativity. A subtle point that is sometimes lost in the AI hype is that the goal of this work is not to \u201creplace\u201d human artists, but to create works in which these genres are made fresh through <\/em><\/strong><\/span>defamiliarization<\/strong><\/span><\/em><\/a>\u2014 Please generate \u201cMore\u00a0almost<\/em> like this, please, but with a strikingly unfamiliar new twist\u201d. Eliciting<\/em><\/strong> and curating<\/em><\/strong> that twist is where the AI artist now puts their effort.<\/p>\n

      Here\u2019s another\u00a0set of examples (2018)<\/a><\/strong>\u00a0by Robbie Barrat, who trained a GAN on a large collection of traditional oil paintings of nudes:<\/p>\n

      \"\"<\/p>\n

      A master of this form is Berlin-based\u00a0Sofia Crespo<\/a><\/strong>, an artist using GANs to generate biological imagery. One of her main focal points is the way organic life uses artificial mechanisms to simulate itself and evolve. Placing great effort into creating custom datasets of biological imagery, she has produced a remarkable body of organic images using GANs.<\/p>\n

      \"Sofia<\/p>\n

      Note that it\u2019s also possible to generate\u00a0music<\/em>\u00a0with GANs. For example, here is\u00a0Relentless Doppelganger<\/em><\/a>\u00a0by DADABOTS<\/strong>\u00a0(CJ Carr and Zack Zukowski)\u2014an infinite live stream of generated death metal:<\/p>\n