For my capstone project, I used Chat GPT, Unity’s Muse chat agent, and Dream Studio for images.
CHAT GPT
I tried to have this help me connect my Quest 2 headset to ESP32 microcontroller via Bluetooth but without it being able to actually see what was happening, I didn’t get very far. Ultimately I decided to have a wired connection instead of Bluetooth.
UNITY’S MUSE
From here, I move to Unity’s chatbot called Muse. I used this one to help me create a custom gravity and it did get me to a place where my ball would stick to the right wall, but it wouldn’t move.
I kept switching between things whenever I hit a wall so when I couldn’t get my gravity to work the way I wanted it to, I started working through serial communication between the ESP32 and Unity. I was using a typical script to establish this connection but I found a blog post that said that certain methods in serial communication in Unity are difficult to work with. I asked Muse to rewrite the script I had so that it would eliminate the aforementioned methods and use something called BaseStream.
Ultimately, none of it worked.
DREAM STUDIO IMAGE GENERATOR
This was definitely much more fun. This generator has some issues with counting and basic geometry but it was not terribly far off spiritually. I liked using the “Analog film” style for the images. Here are some prompts and my favorite results:
A person is standing in front of a boardgame that’s in the shape of an unfolded dodecahedron A person wearing a VR headset is inside of a room with twelve pentagon walls, one of the walls has a maze and another wall has a set of concentric rings
A person wearing a VR headset is inside of a room with twelve pentagon walls A person wearing a VR headset is inside of a dodecahedron-shaped room A trapped space traveler and a personified voice-controlled AI work together to save a spaceship from self-destruction in a black and white, creepy world