Surya Narreddi
Summary
Surya Narreddi and collaborators train a language model with reinforcement learning to paint watercolour hibiscus images by writing editable p5.brush JavaScript rather than emitting pixels.
Code is the editable artefact.
Writing p5.brush sketches instead of pixels lets humans revise the image without returning to the model prompt.
When you make an image with an AI model, the only way to participate is the prompt.
The code is the artefact, and the code is editable.
Aesthetic RL is a reward-design problem.
Verifiable signals work for math and games; subjective quality needs carefully authored criteria or the model converges or drifts.
RL works when the reward is verifiable. A math problem is right or wrong.
Correlated judges waste gradient.
Nine absolute scores measured the same quality repeatedly while length ramps saturated early, locking the model on flat clip-art flowers.
Pairwise taste against a hand-rated pool.
Judging rollouts against love-tier references opened dynamic range, beat the old plateau faster, and compressed winning sketches under 2,000 tokens.
Short allowlists beat long API docs.
GEPA evolved a prompt with eight brush methods and no documentation; long references caused confident, well-formatted API hallucinations.
Long reference documentation in a system prompt made the models hallucinate APIs.
Read the original ↗Save to your Sponge
Summarized with Sponge. Quotes are the author’s own words.