A constraint-aware GPU allocator was benchmarked against a FIFO scheduler across seven scenarios, achieving up to 33 percentage points higher GPU utilization and up to 105% more priority-weighted output on identical hardware. The system treats real-time inference demand as a fluctuating curve rather than a static reservation, allowing batch-like jobs to fill capacity during troughs while respecting strict constraints like contiguous GPU blocks and non-preemption.
- Utilization rose from a 52–85% band to 72–88%, with priority-weighted value increasing by an average of 52% across contended scenarios.
- In a training-heavy workload on 8 GPUs, utilization jumped from 53.6% to 87.0%, more than doubling the output value.
- The allocator operates in 1–2 milliseconds for standard scenarios and 15 milliseconds for large-scale tests with 64 GPUs and 30 jobs.
- Even with uniform priorities, the system improved utilization from 76.8% to 87.5%, demonstrating that horizon planning contributes gains beyond priority weighting alone.
By optimizing allocation decisions across the entire scheduling horizon rather than relying on arrival order, the allocator prevents idle capacity caused by static reservations and ensures high-priority workloads are scheduled efficiently.