AI and Machine Learning Advances

Large-scale machine learning inference environments nowadays are evolving into clusters of non-uniform GPUs with different workload durations, computational needs and submission rates. The study does not test a GPU scheduling algorithm, but rather it performs workload analysis and runtime prediction based on workload traces to present the concept of workload-aware GPU scheduling and to support an AI-informed dynamic GPU scheduling framework. The public dataset with 466,867 machine learning jobs from 84 organizations was processed using descriptive statistics, Spearman correlation, Kruskal–Wallis and Mann–Whitney tests, temporal workload analysis, requested capacity-time measures, and chronological run-time prediction with a HistGradientBoostingRegressor. The result indicated that job durations were highly skewed, GPU-model usage was also unbalanced, and there were significant temporal variations in arrivals, completions, and concurrent demand for GPU models. The run-time was positively associated with CPU request, GPU request, and worker count, with the relationships being generally weak to moderate. The prediction model achieved a positive but low (R2) result, but failed to see a reduction in MAE compared to the median baseline, suggesting low run-time predictability from the available features. These results led to the development of a conceptual AI augmented approach, which brings together workload monitoring, feature extraction, run-time prediction, resource planning and decision support. The study concludes that workload-trace analytics can provide insights for the future capacity planning of GPUs and workload-aware scheduling, but reliable validation requires richer queue, memory, utilization, energy and hardware telemetry data.