Researchers introduce Knowledge--''Less'' Language Models (KLLMs), a training paradigm that anonymizes named entities during pretraining to shift models away from parametric recall and toward evidence-grounded reasoning.
- KLLMs are pretrained on corpora with anonymized named entities, removing entity-linked factual supervision.
- This intervention substantially reduces closed-book factual recall while improving performance on tasks where information is provided as context.
- Across multiple scales, KLLMs outperform baselines on contextual question answering, fact verification, and hallucination detection benchmarks.
- In retrieval-grounded settings with imperfect evidence, KLLMs achieve up to 20--25% relative gains over standard language models.
- The models exhibit better calibration (improved ECE, Brier score, AUROC) and more reliable abstention behavior.
The results demonstrate that suppressing entity-linked supervision induces a shift in epistemic behavior, making KLLMs rely less on parametric knowledge and more on external evidence for improved reliability.