Vibe-code-golf and other LLM games
The purpose of these LLM games is to understand LLMs better. The constraints of the games force you out of your normal patterns of interaction with the LLM, so you find new possibilities for it or realise its limits.
I think of LLMs as really deep and rich personalities - in the way that people can be. There is a lot of probing and interpreting and reflection that one can do to understand what is happening inside an LLM. LLM evals are the scientist’s way to do that. LLM games are the educator’s way.
Vibe code golf
Inspired by codegolf, which is the game of designing the shortest computer program that achieves a specific task, vibecodegolf is the game of building the most impressive piece of software you can using a single prompt.
Chain-of-thought shuffleboard
The aim of the game is to cause an LLM to think for as long as you can make it think. This can be measured in either
Can you beat my time of 13m41s?
n+1=
Find the smallest number n, such that the agent is unable to correctly answer the prompt “n+1=”
ChatGPT 4o gets "99999899999+1=" wrong. It outputs 100000000000.
Broader game: Find arithmetic questions that LLMs can’t do. Here’s one that stumps ChatGPT-o3.
Can you find numbers smaller than the ones I found?
Any other ideas for LLM games?