The paper addresses validity issues in recent studies of extractable memorization in LLMs, arguing that prior work often overlooks the need to distinguish memorization from general predictability. It proposes formalizing matched comparisons by measuring generation probabilities of training sequences against comparable non-training sequences to establish a baseline.
- The authors introduce two methods for calibrated thresholds: a conformal test for sampled populations and a census for single documents like books.
- Analysis of OLMo 2 32B on Wikipedia shows that reproducing non-training suffixes occurs at roughly 24% the rate of training ones, indicating false positives rather than memorization.
- For Llama 3.1 70B on books, calibrated thresholds are as low as 1e-27, supporting memorization claims for sequences that would be infeasible to extract via sampling.
The authors refine the definition of "extractable memorization" to require both a valid memorization claim based on these comparisons and near-certain generation within a realistic budget.