BiRG-LoRA uses rank-gated LoRA for improved medical question answering

Researchers propose BiRG-LoRA, a parameter-efficient adaptation method for medical multiple-choice question answering that dynamically adjusts adapter ranks based on input conditions. The method employs a biaxial gate to select a sparse subset of rank atoms from a single LoRA module per layer, combining semantic evidence with clinical priors.

• Achieves 69.31% macro-average accuracy across CMB, CMExam, MedQA, and MedMCQA benchmarks using Qwen3-8B.
• Outperforms MoELoRA by 0.89 percentage points while utilizing 28.1% fewer trainable parameters.
• Improves over vanilla LoRA r16 and active-rank-matched LoRA r4 by 0.83 macro points.
• Demonstrates robustness to moderate tag noise through evaluation-time weak-axis perturbation checks.

The results indicate that clinically structured rank allocation enhances cross-benchmark medical QA performance under a matched single-seed protocol.