Tool-making pipeline reduces p50 latency by 42% in production LLM agents

The article introduces an agentic tool-making pipeline that replaces inference-time code regeneration with pre-compiled, validated tools for repeated standard operating procedure (SOP) steps. This approach grounds synthesis in the live environment by collecting execution traces and repairing candidate tools against labeled cases before deployment.

• The system compiles repeated SOP steps into versioned tools to avoid regenerating code on every request.
• In a Fulfillment Center alarm-triage system, tool calls reduce p50 latency by 42%.
• End-to-end error rate is reduced by up to 53% on 1,500 historical alarms by suppressing run-to-run variance.
• A controlled ablation shows that returning compact structured verdicts reduces p50 latency by a further 62%.

The authors argue that self-evolving agents make industrial LLM systems faster, more reliable, and easier to operate while improving auditability.