Student Area

I’m interested in learning how to write shaders as well as creating an immersive world using Unity and Blender. My three ideas are either one or the other or a combination of both. I’m not sure if I want to go with writing shaders in Unity however since it might be too difficult to tackle both learning HLSL and getting used to Unity.

Scan of the Month features CT scans of common products, using 3D rendering + scrolling to highlight different components and stories behind their manufacture. It uses Webflow + Lottie but I think something like this would work well in three.js.

The NYTimes is naturally a source of inspiration, and they have done some helpful write-ups + demos using three.js and their own (public) libraries for loading 3D tiles and controlling stories with three.js.

More cool things made with three.js:

• Project overview
  • I want to train my own GAN with images from this Instagram account: subwayhands.
• Related work
  • GANshare: Creating and Curating Art with AI for Fun and Profit
  • Kishi Yuma: GAN trained on pictures of hands

• • Mario Klingemann: Turn finger drawings into a GAN generated image

• • Gloomy Sunday

• Technical Implementation:
  • Instagram Scraping Tools/Tutorials
    • Apify – Instagram Scraper:  I played around with this tool for a bit. Overall, it’s really easy to use. Pros: fast, easy, I don’t need to do much other than inputting the Instagram page link. Cons: It crapes a lot of information, some of which are irrelevant to me. This requires me to filter/clean up the output file.  And is doesn’t directly output the pictures, instead, it provides the display URL for each image, and the URL expires after a certain amount of time.
    • GitHub – Instagram scraper :
      • There is a youtube tutorial showing how to use this.
      •  I simply could not get this one to work on my laptop, though it sees to work for others. Need to play around with this a bit more.
    • Python library  – BeautifulSoup:
      • stack overflow post
      • Will need to parse the rendered with something like Selenium.
    • Alternative method: Manually screenshot pictures
      • Although this is boring, redundant work, it gives me more control on what pictures I want to use to train my GAN.
  • Pix2Pix: If time allows, I want to look into using webcam to record hand posture and generate images based on realtime camera input.

Because I am finishing up previous projects, this post will be about telematic experiences. I want to make a simple game or communication method for friends online, I looked at some apps and websites that I enjoy playing or using with my friends.

One game that me and my friends played a lot during quarantine was Drawphone. Basically, each person in the group draws a prompt given by another person in the group and it continues like the game telephone. A simple drawing game like this brings a lot of joy and laughter, which is something I want to emulate.

Also, I took some inspiration from upperclassmen in design who created a website/application called Pixel Push. The project allows for groups of friends to draw together using their webcams as the paintbrush. They used simple tools like HTML, p5.js, Socket.io, and Google teachable machine to achieve the end result. These are technologies we learned about in class, so I didn’t have to research about them.

For my final project, I am going to use Houdini to make a very short animated scene that involves a character. I hope to use the VR headset as well to be able to quickly sculpt some 3D characters that I can use in the short. I haven’t completely decided yet if I want to bring my Houdini stuff into Unity to make it an interactive project or simply render it in Houdini (maybe both?). I think the most important thing I want to focus on with this project, is just having fun. I’m thinking I’ll end up with something kinda goofy, but we’ll see. At the moment, I want to repurpose this idea I came up with earlier in the semester for this class and make something fun out of it. It will involve using a lot of softbody, springy simulation. I’m not completely sure though if I’m set on this idea.

I’m thinking about making a transformation pipeline that can turn a set of GIS data into a virtual scene, not too different from procedurally generated scenes in games. This will consist of two parts: a transformation that is only done once to generate an intermediate set of data from the raw dataset and another set of code that can render the intermediate data into an interactive scene. I’ll call then stage 1 and stage 2.

To account for the indeterministic nature of my schedule, I want to plan different steps and possible pathways, which are listed below.

Stage 1:

1. Using google earth’s GIS API, start with the basic terrain information for a piece of land and sort out all the basic stuff (reading data into python, python libs for mesh processing, output format, etc.) and test out some possibilities for the transformation.
2. Start to incorporate the RGB info into the elevation info. See what I can do with those new data.
3. Find some LiDAR datasets if point clouds can give me more options.

Stage 2:

1. Just take the code I used for SkyScape (the one with 80k ice particles) and modify it to work with the intermediate format instead of random data.
2. Make it look prettier by using different meshes, movements, and postprocessings that work with the overall concept.
3. Using something like oimo.js or enable3D, add physics to the scene to allow for more user interactions and variabilities of the scene.
4. Enhance user interaction by enhancing camera control, polishing the motions, adding extra interaction possibilities, etc.
5. (If I have substantial time or if Unity is just a better option for this) Learn Unity and implement the above using Unity istead.

I’ll start with modifying the SkyScape code to work with a terrain mesh, which would give me a working product quickly, and go from there.