LRF gateway reduces OOM crashes by forecasting EPO claim workload

A CPU-side Linguistic Resource Forecasting (LRF) gateway is proposed to prevent out-of-memory crashes in distributed LLM schedulers handling statutorily constrained text like EPO claims. By extracting a 16-dimensional text-structure vector and applying an XGBoost predictor, the system forecasts workload traps before GPU memory allocation.

In a 6,000-request live trial, the LRF gateway reduced the operational misroute fraction to 0.087–0.095, significantly lower than the token-count baseline of 0.849. The predictor achieved an AUROC of 0.84, while the dynamic routing threshold yielded an 8.2% relative reduction in misroutes compared to static thresholds. Peak edge VRAM remained bounded at 4.82 GiB despite a 27x variation in WAN delay.

The system routes requests to either a local Qwen2.5-7B worker or a remote ensemble on an NVIDIA H100, ensuring hardware stability under linguistic ambiguity.