The article describes a method for consolidating traffic from over 200 internal applications onto a single self-hosted large language model to address data-residency constraints and GPU fragmentation. The approach closes quality gaps in instruction following, function-calling, and internal task distribution by training separate GRPO experts per axis and merging them via two-stage SLERP.

  • Quality is tracked using offline benchmarks stratified to production traffic and scored by deterministic verifiers or calibrated LLM judges.
  • Separate GRPO experts address specific failure modes: semantic collapse, over-calling, and verbosity hacking.
  • The resulting model surpasses a baseline approximately 7 times larger in total parameters on internal benchmarks.
  • Specific improvements include Arena scores of 69.6 versus 65.8, instruction following of 0.85 versus 0.83, and function-calling of 0.79 versus 0.77.

The consolidated model absorbs 50% of platform traffic, handling 116 million requests per month at a fraction of the serving cost.