A study of four Polish Bielik models (1.5B-11B parameters) demonstrates that unsupervised activation dispersion measures can distinguish known entities from fabricated ones before any answer token is generated. Using inverse participation ratio and spectral entropy on post-SwiGLU MLP activations, the researchers achieved AUROC scores of 0.95-1.00 across athlete, city, writer, and musician domains.
- The signal separates known from fabricated entities at ceiling performance, surviving held-out layer selection and transferring across entity types with mean off-diagonal AUROC of 0.92-0.99.
- While the representational signal for familiarity reaches ceiling at 1.5B parameters, behavioral factual reliability scales sharply, with correct answer counts rising from 0 to 19 as model size increases.
- Separating correct answers from hallucinated ones within known entities remains difficult, with dispersion performing no better than a first-token-entropy baseline.
- Despite this internal awareness of entity familiarity, the models almost never abstain from answering; an audit found only two refusals and one hedge across 2,520 answers.
The findings indicate that entity familiarity and factual reliability are distinct phenomena following different scaling curves, suggesting that internal activation signals alone may not be sufficient to prevent hallucinations without additional mechanisms.