Researchers developed a benchmark to evaluate the multilingual ability of vision-language models (VLMs) to use spatial deictic expressions, such as "this" and "that," across four languages. The study focuses on how well these models jointly reason over language and visual space to ground context-dependent references in an image's spatial structure.
- The benchmark tests VLMs' capacity to select appropriate spatial deictic expressions based on situational context and language-specific spatial distinctions.
- Experiments reveal that tested models use demonstratives differently from humans, particularly regarding the selection of appropriate demonstratives based on the distance to the object.
This evaluation highlights specific gaps in how current VLMs handle complex spatial reasoning tasks compared to human performance.