Researchers introduce CKTN, a new corpus and benchmark designed to address the scarcity of data for Vietnam's ethnic minority languages: Cham, Khmer, and Tay-Nung. The dataset contains 44,367 documents and 24 million subword tokens, covering continued pretraining, category classification, and summary-document retrieval tasks.
- Existing multilingual encoders severely fragment these languages due to differences in script and Vietnamese contact.
- Common adaptation metrics can be misleading, as models may lower language-modeling loss while failing at semantic generalization.
- The authors propose a script-aware adaptation recipe combining vocabulary augmentation with calibrated replaced-token pretraining.
- This approach prevents the discriminator from exploiting trivial script mismatches during training.
The resulting encoder demonstrates substantially less fragmentation and achieves the strongest classification performance among evaluated models, highlighting the limitations of lexical-overlap retrieval as an evaluation signal.