Object-centric LeJEPA uses SAM masks to improve data efficiency

Researchers extend the LeJEPA framework to align object-centric representations rather than whole images, addressing the instability of self-supervised scene partitioning by using off-the-shelf SAM proposals for object masks. The method incorporates an instance-separating loss that treats other objects in a scene as negatives to boost performance.

• Extends LeJEPA's distributional anti-collapse objective to variable-sized sets of objects.
• Uses cheap, off-the-shelf SAM proposals to provide object masks during training.
• Applies an instance-separating loss to treat other objects as negatives.
• Outperforms image-level LeJEPA across two model scales on 10-100% of COCO data.

Object-level LeJEPA demonstrates superior performance in tracking (DAVIS), classification (ImageNet-1k), segmentation (ADE20k), and re-identification (NAVI) compared to its image-level counterpart.