VALIDATING RESOURCE SCHEDULING ALGORITHMS USING REAL GENAI CLUSTER LOAD
Deepak Yadav, Savita Kumari Sheoran
Abstract: GenAI workloads served on GPU clusters exhibit great variance of runtime and bursty arrival patterns, and heterogeneous resource demands mean efficient and fair scheduling becomes more challenging to achieve. This paper looks at a production-grade trace—called cluster-trace-v2026-GenAI—that captures over 1.8 million LoRA-based inference requests along with per-container GPU duty cycle and memory usage at fine detail as its comparative tool and runs a variety of resource scheduling algorithms against it. We show with our trace-driven simulator how first-come-first-served (FCFS), prediction-aware shortest-job-first (SJF), priority-based and fair-share schedulers can be compared on three dimensions: job completion time (JCT), user-level fairness and aggregate resource utilization. Our results show that prediction-aware SJF leads to a mean JCT drop of 23.7% compared to FCFS for the GenAI work, with Jain's fairness index dropping from 0.91 to 0.82 and increased GPU usage as high as 5% during the peak times. Cross-dataset comparison with a previously observed batch/HPC trace confirms that high runtime variance and burstiness of GenAI traffic substantially increases the relative benefit of sophisticated prediction-aware scheduling (mean JCT improved by 12.3% vs. 23.7%). The results imply a need for production GenAI cluster to work on runtime-aware, fairness-constrained schedulers and to refine the prediction ability and policy parameters in terms of striking the trade-off in terms of latency, fairness, and resource efficiency.
Keywords: GenAI; cluster scheduling; Runtime prediction; GPU scheduling; Resource management
DOI: 10.24874/PES08.03.029
Recieved: 02.06.2026 Revised: 23.07.2026 Accepted: 19.08.2026
UDC:
Reads: 3 
