Researchers propose VocaDet, a framework for open-vocabulary object detection and segmentation that learns concepts directly from user-provided positive and negative samples without requiring model retraining. The system transforms continuous visual representations into discrete visual tokens using DINOv3 and agglomerative clustering, storing them in an expandable vector database for efficient retrieval-based recognition.
- Utilizes DINOv3 as the visual feature extractor to generate multi-granularity visual tokens via agglomerative clustering with adaptive sensitivity.
- Stores position-debiased representations and spatial topology information as expandable object memories in a vector database.
- Implements a background filtering mechanism to remove frequent background patterns and reduce redundant retrieval operations in fixed-camera scenarios.
- Demonstrates effective open-vocabulary detection performance on the UA-DETRAC dataset without conventional detector training.
VocaDet supports continuously expandable recognition capability as additional samples are accumulated, addressing scalability issues found in text-prompt or limited-example approaches.