After watching a bunch of Unity tutorials and poking around in the editor a little bit, I’ve decided to learn the engine.

On the Unity side, I’ll either follow a tutorial or just study a sample project to make a first-person walk-around. (First Person Character Controller, Procedural Mesh.) This likely will not include using any other assets, prefabs, or other functions. Just a character controller object that slides along the terrain.

On the python mesh generation side, I’ll be using the open-elevation API to get the elevation data of a given region. I might also try the sentinelsat library to query the satellite map, which I could use for color or some other purposes. I’ll then use pymesh to make a .obj file to use in Unity.

One of my biggest worries was that I would need to find all the individual piano keys to make my notes, good thing the university of Iowa has all the notes from C0 to C8 (https://theremin.music.uiowa.edu/MISpiano.html). The difficulty now will just be in modifying the notes for them to last different lengths.

I also found a metronome implementation: https://gist.github.com/bzgeb/c298c6189c73b2cf777c.
And after those two minutes of searching this github page showed me another way Unity tracks time using AudioSettings.dspTime which is interesting.

With these two things already done for me I think completing the project will be much easier.

I looked into the custom engine behind the indie game “Noita” called “Falling Everything”. This engine creates a fully simulated world of pixels with solids, powders, rigidbodies, liquids, gases, fire, and various oddly-behaving materials. They discussed how everything in the game works under the same fundamental algorithm that effectively generates the behavior of sand. I also learned that I will have to keep my physics simulation a lot more simple due to the amount of optimizations that they put in place for the engine to run effectively that I will be unable to do in unity.

I also played around with the orbit simulator here: https://academo.org/demos/orbit-simulator/  to get an idea of what the orbit section of what I want to create looks like in a 2 body simulation.

While looking up inspiration for my project, I was trying to see if I could find some art piece that was not only made using the Looking Glass, but also one that was interactive. In my search, I found this virtual reality painting simulator that showcases painting strokes made real-time made by another person using an Oculus. This is more of a drawing game/apparatus than a “finished artwork”, but I still find it inspiring because it is an interactive work made using a Looking Glass.

The main creator I have been looking at as I am ironing out the kinks of Unity is the YouTuber Sebastian Lague. He has a ton of videos that detail how he creates small but interesting projects in Unity, such as creating a flight simulator that uses real-word Earth data (such as Earth depth data to create a normal map for the globe model), or creating a barebones yet interconnected simulated  ecosystem.  I found his series on creating procedural plants particularly helpful. I don’t think any of these projects would necessarily be super engaging to play or are super artistically interesting, but I don’t think they aren’t really meant to be. I think they function well to highlight what can be done in Unity and to get viewers excited to create, which I think is really all I need for this project. I have also been watching a few videos on crowd simulation in Unity because I have a few ideas on how I can use a lot of AI agents in my project, which has been useful, but most videos detail projects that are probably more complex than I’ll need.

Right now I am taking this short project centered course taught by the animator Christopher Rutledge, who uses Houdini as his main tool for animation. The course just goes through the entire character development process specific to Houdini, which covers some things I didn’t know yet like KineFX, Houdini’s rigging toolset.

Creating a Mega Character – made in Houdini 19 for Intermediate by Christopher Rutledge at SideFX

He has also used the vr sculpting workflow for character creation.

The awesome thing about Houdini is I can do all of the normal character development process, but also use the simulation power of Houdini to computationally make the scenes/characters themselves more interesting and fun. For example this clip below showcases a really charming character design and added physics simulation to make the movement that much more appealing/charming, funny and unexpected.

 

View this post on Instagram

 

A post shared by christopher rutledge (@tokymegz)

Another resource I use HEAVILY, is Entagma. Their tutorials are incredible. Here is an example of an effect I could create to add onto a character.

 

View this post on Instagram

 

A post shared by Entagma (@therealentagma)

Relevant media – I’m interested in how to explore generativity within nature and how to achieve that using shaders. While the following results most likely aren’t achievable given my current knowledge and time frame of the project, I thought it was relevant to what I want to explore.

Harry Alisavakis

Sakura Bliss 

Technologies and techniques – I’m still unsure whether to pursue only learning GLSL or learning HLSL + Unity, so I included resources for both.

Freya Holmer – Shaders for Game Devs series – I’ve tried to watch this for the past year and I haven’t gotten past the first video

Roystan Unity shaders

Char Stiles’ shader workshop page + Book of Shaders

Scan of the Month features CT scans of common products, using 3D rendering + scrolling to highlight different components and stories behind their manufacture. It uses Webflow + Lottie but I think something like this would work well in three.js.

The NYTimes is naturally a source of inspiration, and they have done some helpful write-ups + demos using three.js and their own (public) libraries for loading 3D tiles and controlling stories with three.js.

More cool things made with three.js: