An engineer has open-sourced a reproduction of Surya Narreddi’s viral project, which trains a coding model to paint watercolours using JavaScript and reinforcement learning. The implementation utilizes the TRL library and OpenEnv to create an end-to-end pipeline on Hugging Face for training, scoring, and inference.
- The system uses p5.brush to simulate watercolour effects like pigment bleeding and paper texture through a restricted set of ten methods.
- Rewards are derived from two judges: HPSv3 for aesthetic preference and a pairwise judge comparing outputs against a hand-rated pool of 178 generated images.
- Three training runs were conducted with varying weights between the judges, demonstrating that the model can learn to imitate specific artistic styles encoded in the dataset.
- The entire recipe, including the RL environment, scorer model, and trained models, is published on the Hugging Face Hub.
This project demonstrates how reinforcement learning over subjective taste can steer a smaller model to produce distinct artistic styles rather than statistically average images.