Researchers present SocialRL, a training recipe that directly reinforces social reasoning in small language models to improve their performance as strategic negotiators. Applied to a 4B model across six domains including Deal-or-No-Deal and Job Interview, the approach enables the model to match or exceed GPT-5 family performance on held-out scenarios.
- In-domain training closes 73-122% of the baseline-to-frontier gap, with 78% of buyer openings anchoring below target versus 3% for untrained models.
- Cross-domain transfer depends on game structure, where structurally paired games lift each other while isolated games do not transfer.
- Cascade RL and multi-teacher on-policy distillation consolidate specialists into a unified 4B model achieving 0.627 average utility, surpassing GPT-4.1 (0.625) and GPT-5.2 (0.613).
- Distilling theory-of-mind traces rather than actions alone improves utility and generalization, with next-action prediction being the key skill for outcome prediction.
The study demonstrates that small models can achieve frontier-level negotiation capabilities through direct social reasoning training and effective cross-domain consolidation strategies.