Co-VGGT uses VGGT's layer specialization to improve co-visibility classification

Researchers demonstrate that the geometric foundation model VGGT implicitly encodes co-visibility, with early layers building 3D-aware representations and late layers acting as dedicated reasoners. Specifically, layer L17 serves as a negative anchor for non-co-visible pairs, providing evidence of layer specialization.

Building on this finding, the authors introduce Co-VGGT, which freezes VGGT and trains a lightweight layer-wise mixture-of-experts head with fewer than 7.5M parameters to classify co-visibility from RGB images alone.

On the Co-VisiON benchmark, Co-VGGT surpasses the human annotation baseline, improving pairwise results by more than 25% and multiview results by 10%. Its predictions are well-calibrated (ECE=0.030), allowing direct use as edge weights in visibility graphs for SfM and SLAM pipelines without post-hoc correction.