ELSA3D introduces elastic semantic anchoring for efficient unified 3D understanding and generation

Researchers introduce ELSA3D, a unified 3D foundation model that addresses implicit text-3D interactions by structuring language and geometric reasoning along matched abstraction scales. The model utilizes a scale-aware octree tokenizer and sparse Anchor Tokens to route semantic cues to relevant 3D scales, ensuring precise cross-modal alignment.

• ELSA3D employs a lightweight per-block router to dynamically determine which text tokens instantiate anchors at specific geometric scales.
• This approach concentrates cross-modal capacity where alignment is most needed while maintaining sparse yet precise interaction.
• The method achieves state-of-the-art performance in image-to-3D generation, text-to-3D generation, and 3D captioning.
• It roughly halves FLOPs and inference latency compared to the non-elastic version of the same model.

ELSA3D outperforms the strongest unified baseline while significantly reducing computational costs, offering a more efficient solution for unified 3D tasks.