Author: CrispySalmon

Training GAN with runwayML

subwayHands

• • Process:
    1. Instagram image scrape using selenium and bs4.
       • This part took me the most time. Instagram makes it really difficult to scrape, I had to make several test dummy accounts to avoid having my own account banned by Instagram.
       • Useful Tutorial: https://www.youtube.com/watch?v=FH3uXEvvaTg&t=869s&ab_channel=SessionWithSumit
    2. Image cleanup, crop(square) and resize(640*640px)
       • After I scrapped this ig account’s entire page, I manually cleaned up the dataset to get rid of any images with face, more than two hands or no hands at all. And the remaining 1263 pictures are cropped and resized to be better processed by runwayML.
    3. Feed dataset to runwayML and let it do its magic.
  • Result: Link to google folder Link to runwayML Model
    • When looking at the generated images zoomed out, thy defiantly share a visual proximity with the ig:subwayhands images.

However, looking at the enlarged generated images, it’s clearly not quite at the level of actual hands. You can vaguely make out where the hand will be and what’s the posture of the hand/hands.

My Hair

• • process:
    1. scan my hairs:
       • 
       • I had this notebook full of my hairs that I collected throughout my freshmen year. I scanned them in to unified(1180px*1180px) squares, and at the end I was able to get an image dataset with 123 images of my own lost hairs.
       • 
    2. Feed dataset to runwayML and let it do its magic.
  • Result: Link to google folder Link to runwayML model
    • 8bcb94f4-dc99-46c9-9190-d680a6c14ecc
    • 
    • Despite the small size of this dataset, the generated images exceeded my expectations and seems to more successfully resemble an actual picture of shower drain hair piece. This is probably because the hair dataset is much more regularized compared to the subway hands dataset.  The imagery is also much simpler.

• Project overview
  • I want to train my own GAN with images from this Instagram account: subwayhands.
• Related work
  • GANshare: Creating and Curating Art with AI for Fun and Profit
  • Kishi Yuma: GAN trained on pictures of hands

• • Mario Klingemann: Turn finger drawings into a GAN generated image

• • Gloomy Sunday

• Technical Implementation:
  • Instagram Scraping Tools/Tutorials
    • Apify – Instagram Scraper:  I played around with this tool for a bit. Overall, it’s really easy to use. Pros: fast, easy, I don’t need to do much other than inputting the Instagram page link. Cons: It crapes a lot of information, some of which are irrelevant to me. This requires me to filter/clean up the output file.  And is doesn’t directly output the pictures, instead, it provides the display URL for each image, and the URL expires after a certain amount of time.
    • GitHub – Instagram scraper :
      • There is a youtube tutorial showing how to use this.
      •  I simply could not get this one to work on my laptop, though it sees to work for others. Need to play around with this a bit more.
    • Python library  – BeautifulSoup:
      • stack overflow post
      • Will need to parse the rendered with something like Selenium.
    • Alternative method: Manually screenshot pictures
      • Although this is boring, redundant work, it gives me more control on what pictures I want to use to train my GAN.
  • Pix2Pix: If time allows, I want to look into using webcam to record hand posture and generate images based on realtime camera input.