Study identifies FragileTokens that fail contextual copying despite passing isolation probes
A study characterizes "FragileTokens," vocabulary entries in open-weight language models that successfully copy when isolated but exhibit errors when embedded in surrounding text. The research highlights that literal identity preservation is not guaranteed by standard isolation tests, as tokens can be deleted, substituted, or truncated within sequences.