Researchers at NVIDIA have introduced Spatial-IQ, a diagnostic framework designed to deconstruct the spatial intelligence of multimodal large language models (MLLMs). Unlike existing benchmarks that treat models as black boxes, this approach decomposes object counting in stacked 3D structures into nine perceptual and cognitive sub-tasks aligned with human developmental stages.
- The framework utilizes NVIDIA Isaac Sim to procedurally generate a dataset of approximately 80,000 stacked 3D structures with ground truth for each sub-task.
- Evaluations across free-response text, multiple-choice images, and image editing formats reveal that top-performing models often succeed at target tasks while failing lower-level sub-tasks, indicating shortcut behavior.
- The study demonstrates that training with chain-of-thought supervision over these hierarchical sub-tasks, combined with reinforcement learning with verifiable rewards, significantly improves both spatial consistency and target-task accuracy.
This decomposition serves as both a diagnostic tool to identify specific failure modes and an effective training signal for enhancing the robustness of spatial reasoning in MLLMs.