CMDR introduces a benchmark and framework for contextual multimodal document retrieval

The authors introduce CMDR and CMDR-Bench, a new task and benchmark designed to evaluate multimodal document retrieval by requiring the modeling of document context rather than just lexical or semantic matching. To address this, they propose CMDR-Embed, a framework that jointly encodes multiple pages to derive contextual page-level embeddings, trained with a novel CMCL objective.

• CMDR-Bench evaluates retrieval methods on their ability to aggregate information across multiple pages.
• CMDR-Embed explicitly incorporates document context by jointly encoding multiple pages.
• CMCL is a contextual multimodal contrastive learning objective that balances contextual modeling with page-level discriminability.
• Experiments show that CMDR-Embed significantly outperforms non-contextual embeddings.

The work highlights the importance of context-aware multimodal embeddings for advancing document retrieval tasks.