This is an automated archive made by the Lemmit Bot.

The original was posted on /r/artificial by /u/Desert_Trader on 2024-10-26 18:29:24+00:00.

I want to hook up ChatGPT to control my outdated but ahead of its time WOWWEE Rovio. But until I remember how to use a soldering iron, I thought I would start small.

Using ChatGPT to write 100% of the code, I coaxed it along to use an ESP32 embedded controller to manipulate a 256 LED Matrix “however it wants”.

The idea was to give it access to something physical and “see what it would do”.

So far it’s slightly underwhelming, but it’s coming along ;)

The code connects to WiFi and the ChatGPT API to send a system prompt to explain the situation “You’re connected to an LED matric to be used to express your own creativity.” The prompt gives the structure of commands on how to toggle the led’s including color, etc. and lets it loose to do whatever it sees fit.

With each LED command is room for a comment that is then echo’d to serial so that you can see what it was thinking when it issued that command. Since ChatGPT will only respond to prompts, the controller will re-prompt in a loop to keep it going.

Here is an example of some (pretty creative) text that it adds to the comments…

Comment: Starting light show.
Comment: Giving a calm blue look.
Comment: Bright green for energy!
Comment: Spreading some cheer!
Comment: Now I feel like a fiery heart!
Comment: Let's dim it down.
Comment: A mystical vibe coming through.
Comment: Ending my light show. 

And here is the completely underwhelming output that goes along with that creativity:

For some reason, it likes to just turn on then off a few lights in the first 30 or so of the matrix followed by a 100% turn on of the same color across the board.

I’m going to work on the prompt that kicks it off, I’ve added sentences to it to fine tune a bit but I think I want to start over and see how small I can get it. I didn’t want to give it too many ideas and have the output colored by my expectations.

Here are two short videos in action. The sequence of blue lights following each other was very exciting after hours of watching it just blink random values.

Looking forward to getting (with a small prompt) to do something more “creative”. Also looking forward to hooking it up to something that can move around the room!

All in all it took about 6 hours to get working and about $1 in API credit. I used o1-preview to create the project, but the controller is using 4o or 4o-mini depending on the run.