关键词: 异构GPU集群, 深度Q网络, 批次作业, 多目标优化

Abstract: Heterogeneous GPU clusters have become a core support in the field of high-performance computing, providing essential computational power for deep learning, scientific computing, and other batch processing tasks. However, batch jobs exhibit significant diversity in resource requirements, meaning that jobs within the same batch may have vastly different GPU and CPU demands. This heterogeneity and diversity in resource requirements significantly increase the complexity of resource allocation, resulting in low overall resource utilization (only 25%~50%), prolonged job waiting times, and worsening load imbalance, which severely constrain the improvement of Service Quality (QoS). Traditional resource allocation strategies, such as random allocation (Random), Round-Robin scheduling (RR), and Shortest Job First (SJF), primarily rely on static rules or heuristic methods, making them ineffective adapt to the complexity and dynamic nature of heterogeneous computing resources. These approaches often lead to resource wastage, unbalanced scheduling, and decreased system throughput. To address these challenges, this paper proposes a multi-objective optimization-based resource allocation method for heterogeneous GPU clusters, aiming to improve resource utilization, reduce job waiting time, and optimize overall load balancing, thereby enhancing cluster computing efficiency and QoS.
This study formulates the batch job resource allocation problem in heterogeneous GPU clusters as a multi-objective optimization problem, considering GPU/CPU resource utilization, job waiting time, and load balancing as key optimization objectives. To enable efficient decision-making, this paper constructs a Markov decision process framework that includes state space, action space, and reward functions. Furthermore, a multi-objective optimization-based resource allocation strategy is proposed using deep reinforcement learning techniques. Specifically, the paper adopts the Deep Q-Network (DQN) for decision-making optimization, leveraging Deep Neural Networks (DNN) to learn optimal scheduling policies. Additionally, a dual-parameter threshold linear decay strategy is incorporated to dynamically balance exploration and exploitation, ensuring efficient scheduling strategy optimization in complex heterogeneous environments.
This study builds an experimental platform using Python and TensorFlow and simulates a heterogeneous cluster environment with three types of GPU nodes: T1, T2, and T3. The experimental data is sourced from a real Alibaba Cloud dataset, and systematic evaluations are conducted under various job loads and resource configurations. Furthermore, to conduct a comprehensive analysis of model performance, this study performs a sensitivity evaluation of six different DNN architectures (with fixed input and output layers), focusing on the impact of the number of hidden layers and activation functions. To systematically assess the effectiveness of the dual-parameter threshold linear decay strategy in optimizing the exploration-exploitation trade-off, three comparative experiments are designed: the first adopts a fixed exploration rate, the second employs a dual-parameter exponential decay strategy, and the third utilizes a single-parameter threshold linear decay strategy. These experiments provide a comparative analysis of different exploration strategies and their influence on model learning performance.
The experimental results demonstrate that the proposed method effectively stabilizes GPU and CPU utilization at 65%~70% and 70%~75%, respectively, while significantly reducing the average job waiting time from over 10 seconds to approximately 4 seconds. Additionally, it improves load balancing across nodes. A generalization evaluation is conducted across different batch job sizes (200, 500, and 1000 tasks) and heterogeneous cluster configurations (with 1 to 3 types of GPUs), revealing that the proposed method consistently outperforms traditional resource allocation algorithms (Random, RR, SJF) in overall performance, demonstrating strong generalization capability and robustness. Compared to single-objective optimization strategies, the proposed multi-objective optimization model achieves an average improvement of 9.52% in resource utilization, a 131.78% reduction in average job waiting time, and a 16.66% reduction in load imbalance variance, effectively optimizing both scheduling efficiency and fairness in heterogeneous cluster environments. In terms of computational overhead, the DQN-based deep reinforcement learning algorithm not only requires lower training costs but also offers faster decision-making, outperforming common deep reinforcement learning approaches such as DDQN, PPO, and A3C.
Although the proposed method exhibits outstanding performance in multi-objective optimization, future research can explore dynamic weight adjustment mechanisms and scalability for large-scale clusters. Potential directions include integrating federated learning to achieve distributed scheduling and supporting preemptive job scheduling scenarios.

Key words: heterogeneous GPU cluster, deep Q-network, batch jobs, multi-objective optimization