SeRIn decomposes multimodal fusion for sentiment analysis

Researchers propose SeRIn (Segregate, Refine, Integrate), a multimodal language model fusion scheme that enforces the separation of modality-specific signal refinement and cross-modal interaction as an architectural prior.

• Modality-specific representations evolve along isolated pathways, refined against their respective encoder context.
• A dedicated cross-modal pathway accumulates joint evolution without contaminating unimodal streams.
• Full cross-modal interaction is deferred to a final prediction step.
• Ablations confirm that structured interactions, not added capacity, drive performance gains.

SeRIn achieves state-of-the-art results on the CH-SIMS and CMU-MOSEI benchmarks, improving all metrics on both datasets.