QLoRA fine-tuning of Gemma 4 31B closes cross-lingual relation extraction gap for Romanian

The study investigates cross-lingual relation extraction for Romanian by translating the SemEval-2010 Task 8 benchmark and evaluating Gemma 4 31B against encoder baselines. Results show that QLoRA fine-tuning improves macro F1-Score by more than 22 percentage points, reducing the cross-lingual gap from 3.3 to 1.4pp.

• Romanian incurs a 3 to 5 pp drop relative to English in prompt-only settings.
• Few-shot prompting provides only marginal gains over zero-shot.
• Encoder baselines come within 1-4pp of QLoRA Gemma despite being 50-250 times smaller.
• Monolingual Romanian BERT at 125M parameters matches multilingual XLM-R at 278M.

The authors conclude that using a 31B model for single-task RE on Romanian is weak in deployment scenarios where compute matters.