Multimodal RLHF alignment improves Han-Nom to Vietnamese translation

Researchers propose a multimodal RLHF preference-alignment framework for translating degraded Han-Nom manuscripts into modern Vietnamese, addressing challenges like rare logographic characters and limited parallel supervision. The model integrates visual features from CLIP ViT-L/14@336 with text representations from bert-base-chinese and vinai/phobert-base, compressing them via a fusion block before generation.

• DPO achieves the best BLEU-4, ROUGE-L, BERTScore, semantic similarity, CER, WER, and token accuracy compared to PPO and KTO.
• PPO obtains the highest precision, recall, and F1 scores among the tested methods.
• KTO remains competitive through its desirable-undesirable utility objective.
• All preference-aligned policies improve BLEU-4 and semantic-similarity scores over the supervised fine-tuned baseline.

These results indicate that multimodal preference optimization complements supervised learning by improving lexical and semantic quality in low-resource historical translation.