Hallucination Self-Play bootstraps detector via evolved generator

Researchers introduce Hallucination Self-Play (HSP), a framework that allows a hallucination detector to bootstrap using an evolved generator. The method addresses the scarcity of high-quality annotated data by enabling iterative improvement rather than treating the generator as static.

The framework initializes both roles from the same base model: a detector for assessing faithfulness and a generator producing hard-to-detect hallucinations. The detector is fine-tuned on human-labeled data and used as a reward model to train the generator via reinforcement learning from AI feedback (RLAIF). The evolved generator then synthesizes hallucination data to further optimize the detector through rule-based reinforcement learning.

Experiments on the RAGTruth benchmark show that HSP can progressively enhance a small LLM to match or outperform advanced LLMs without external supervision.