Carbon-Aware Kubernetes Scheduling Using Deep Reinforcement Learning for Mixed Batch and Latency-Sensitive Workloads
DOI:
https://doi.org/10.15680/IJCTECE.2024.0703009Keywords:
Carbon-aware scheduling, deep reinforcement learning, Proximal Policy Optimization, Kubernetes, green cloud computing, carbon intensity, Markov Decision Process, emission variance reductionAbstract
Carbon emissions from cloud data centers vary significantly with workload placement owing to regional and temporal differences in carbon electricity intensity. Mainstream Kubernetes schedulers remain entirely agnostic to these differences, systematically missing opportunities for emission reduction. This paper presents a carbon-aware Kubernetes scheduling framework driven by Proximal Policy Optimization (PPO), a deep reinforcement learning algorithm chosen for its stable policy gradient updates under non-stationary carbon signals. The scheduling problem is formulated as a Markov Decision Process in which the agent jointly optimizes carbon efficiency, SLA compliance, and resource utilization. Workloads are classified into latency-sensitive services and delay-tolerant batch jobs, with asymmetric reward weighting that strongly penalizes SLA violations for the former while prioritizing carbon placement for the latter. Large-scale simulation and Azure Kubernetes Service testbed experiments show that the proposed scheduler reduces mean carbon emissions per workload by 28% compared with the default scheduler and by 15% compared with a rule-based carbon-and-SLA-aware heuristic, while holding SLA violation rates below 1.2%. Critically, the PPO scheduler also reduces carbon emission variance by 49% under stochastic conditions, providing more predictable environmental performance than any baseline—a property with direct operational value for sustainability commitments.
References
[1] T. Zhao, T. Piontek, K. Haghshenas, and M. Aiello, "Carbon emission-aware job scheduling for Kubernetes deployments," The Journal of Supercomputing, vol. 80, no. 1, pp. 549–569, 2023. https://doi.org/10.1007/s11227-023-05506-7
[2] D. Zhao and J. Zhou, "An energy and carbon-aware algorithm for renewable energy usage maximization in distributed cloud data centers," Journal of Parallel and Distributed Computing, vol. 165, pp. 156–166, 2022. https://doi.org/10.1016/j.jpdc.2022.04.001
[3] T. Le and D. Wright, "Scheduling workloads in a network of datacentres to reduce electricity cost and carbon footprint," Sustainable Computing: Informatics and Systems, vol. 5, pp. 31–40, 2014. https://doi.org/10.1016/j.suscom.2014.08.012
[4] E. Khodayarseresht, A. Shameli-Sendi, Q. Fournier, and M. Dagenais, "Energy and carbon-aware initial VM placement in geographically distributed cloud data centers," Sustainable Computing: Informatics and Systems, vol. 39, p. 100888, 2023. https://doi.org/10.1016/j.suscom.2023.100888
[5] A. Radovanovic et al., "Carbon-aware computing for datacenters," IEEE Transactions on Power Systems, vol. 38, no. 2, pp. 1270–1280, 2022. https://doi.org/10.1109/tpwrs.2022.3173250
[6] J. Yan et al., "Energy-aware systems for real-time job scheduling in cloud data centers: A deep reinforcement learning approach," Computers & Electrical Engineering, vol. 99, p. 107688, 2022. https://doi.org/10.1016/j.compeleceng.2022.107688
[7] Z. Wang et al., "Reinforcement learning based task scheduling for environmentally sustainable federated cloud computing," Journal of Cloud Computing, vol. 12, no. 1, 2023. https://doi.org/10.1186/s13677-023-00553-0
[8] W. Liu et al., "Online job scheduling scheme for low-carbon data center operation: An information and energy nexus perspective," Applied Energy, vol. 338, p. 120918, 2023. https://doi.org/10.1016/j.apenergy.2023.120918
[9] S. Zhang, M. Xu, W. Y. B. Lim, and D. Niyato, "Sustainable AIGC workload scheduling of geo-distributed data centers: A multi-agent reinforcement learning approach," arXiv:2304.07948, 2023.
[10] R. Kaur et al., "An advanced job scheduling algorithmic architecture to reduce energy consumption and CO2 emissions in multi-cloud," Electronics, vol. 12, no. 8, p. 1810, 2023. https://doi.org/10.3390/electronics12081810
[11] T. Thein, M. M. Myo, S. Parvin, and A. Gawanmeh, "Reinforcement learning based methodology for energy-efficient resource allocation in cloud data centers," Journal of King Saud University – Computer and Information Sciences, vol. 32, no. 10, pp. 1127–1139, 2018.
[12] T. Subramanian, "Carbon-aware scheduling for serverless computing," M.S. thesis, Technical University of Munich, 2023.
[13] M. Chadha et al., "GreenCourier: Carbon-aware scheduling for serverless functions," in Proc. ACM Middleware, 2023, pp. 18–23. https://doi.org/10.1145/3631295.3631396
[14] S. Anasuri and K. K. Pappula, "Green HPC: Carbon-aware scheduling in cloud data centers," International Journal of Emerging Research in Engineering and Technology, vol. 4, 2023.
