SVF-CR: Synchronized Visual-Facial Cross-Refinement for Multimodal Ambivalence and Hesitancy Recognition

Researchers propose SVF-CR, a framework for recognizing ambivalence and hesitancy by synchronizing visual and facial evidence. The method extracts aligned whole-video and cropped-face tokens, refining them through intra-modal self-attention and bidirectional cross-attention.

• Synchronized tokens are mutually refined before constructing segment-level evidence via consistency and discrepancy modeling.
• Textual and acoustic features undergo context self-attention and fuse with visual-facial evidence at the decision stage.
• Experiments on the BAH public evaluation split achieve a macro-F1 of 0.7156, outperforming global token fusion baselines.

The approach improves recognition accuracy by modeling how temporally aligned behavioral evidence interacts across modalities.