The authors introduce CtrlVTON, a framework that addresses the lack of user control in virtual try-on by recasting the task as image editing with pixel-level segmentation masks. This approach allows users to specify garment size, style, and spatial placement on the body.
- The system defines Visual-Instance-Prompt Segmentation via VIP-SAM to segment specific garment instances from flatlay images onto a person.
- CtrlVTON uses these masks to control layout details such as whether clothing is tucked in or open.
- Both VIP-SAM and CtrlVTON achieve state-of-the-art results on their respective tasks.
- The generated images follow user-provided layouts more faithfully than proprietary editing systems while matching them on garment fidelity.
This work provides a method for realistic garment transfer with precise control over how the clothing is worn.