Researchers introduce NeoMME, a family of 260M and 800M multilingual multimodal encoders that process text and raw image patches using a single bidirectional Transformer. Unlike generative visual language models, NeoMME does not rely on separate pretrained vision towers or causal decoders, training the entire model from scratch with a masked discrete-diffusion objective.

  • NeoMME-Retriever fine-tuned for visual document retrieval returns both dense and late-interaction embeddings in one forward pass.
  • The 260M model achieves an nDCG@10 of 0.523 on ViDoRe v3, lying on the Pareto frontier while using significantly fewer parameters than comparable models.
  • Hierarchical token pooling and asymmetric quantization reduce late-interaction index storage from roughly 1.5 MB to 6 kB per page while retaining over 95% of baseline retrieval quality.
  • On an NVIDIA L40S GPU, the 260M model encodes about 51 pages per second at 2048×2048 resolution, offering nearly twice the throughput of ColModernVBERT.

The models are available in Hugging Face Transformers under the Apache 2.0 license, enabling efficient indexing and search for multimodal corpora without OCR preprocessing.