Sentence Transformers version 6 introduces the MultiVectorEncoder class, enabling ColBERT-style late interaction retrieval directly within the library without requiring external dependencies like PyLate. This addition allows users to load multi-vector checkpoints from various sources, including PyLate and Stanford-NLP formats, using a unified API.

  • Multi-vector models preserve token-level information by keeping one vector per token, scoring queries against documents via the MaxSim operator to improve retrieval quality over single-vector dense embeddings.
  • The implementation supports visual document retrieval through colpali-engine models and handles various checkpoint formats natively.
  • Indexing strategies such as fast-plaid are utilized to manage the increased storage costs associated with storing vectors for every token.

This feature brings late interaction capabilities into the core Sentence Transformers ecosystem, allowing users to leverage state-of-the-art retrieval methods while maintaining compatibility with existing dense and sparse model workflows.