Abstract: Heterogeneous GPU clusters have become a core support in the field of high-performance computing, providing essential computational power for deep learning, scientific computing, and other batch processing tasks. However, batch jobs exhibit significant diversity in resource requirements, meaning that jobs within the same batch may have vastly different GPU and CPU demands. This heterogeneity and diversity in resource requirements significantly increase the complexity of resource allocation, resulting in low overall resource utilization (only 25%~50%), prolonged job waiting times, and worsening load imbalance, which severely constrain the improvement of Service Quality (QoS). Traditional resource allocation strategies, such as random allocation (Random), Round-Robin scheduling (RR), and Shortest Job First (SJF), primarily rely on static rules or heuristic methods, making them ineffective adapt to the complexity and dynamic nature of heterogeneous computing resources. These approaches often lead to resource wastage, unbalanced scheduling, and decreased system throughput. To address these challenges, this paper proposes a multi-objective optimization-based resource allocation method for heterogeneous GPU clusters, aiming to improve resource utilization, reduce job waiting time, and optimize overall load balancing, thereby enhancing cluster computing efficiency and QoS.
This study formulates the batch job resource allocation problem in heterogeneous GPU clusters as a multi-objective optimization problem, considering GPU/CPU resource utilization, job waiting time, and load balancing as key optimization objectives. To enable efficient decision-making, this paper constructs a Markov decision process framework that includes state space, action space, and reward functions. Furthermore, a multi-objective optimization-based resource allocation strategy is proposed using deep reinforcement learning techniques. Specifically, the paper adopts the Deep Q-Network (DQN) for decision-making optimization, leveraging Deep Neural Networks (DNN) to learn optimal scheduling policies. Additionally, a dual-parameter threshold linear decay strategy is incorporated to dynamically balance exploration and exploitation, ensuring efficient scheduling strategy optimization in complex heterogeneous environments.
This study builds an experimental platform using Python and TensorFlow and simulates a heterogeneous cluster environment with three types of GPU nodes: T1, T2, and T3. The experimental data is sourced from a real Alibaba Cloud dataset, and systematic evaluations are conducted under various job loads and resource configurations. Furthermore, to conduct a comprehensive analysis of model performance, this study performs a sensitivity evaluation of six different DNN architectures (with fixed input and output layers), focusing on the impact of the number of hidden layers and activation functions. To systematically assess the effectiveness of the dual-parameter threshold linear decay strategy in optimizing the exploration-exploitation trade-off, three comparative experiments are designed: the first adopts a fixed exploration rate, the second employs a dual-parameter exponential decay strategy, and the third utilizes a single-parameter threshold linear decay strategy. These experiments provide a comparative analysis of different exploration strategies and their influence on model learning performance.
The experimental results demonstrate that the proposed method effectively stabilizes GPU and CPU utilization at 65%~70% and 70%~75%, respectively, while significantly reducing the average job waiting time from over 10 seconds to approximately 4 seconds. Additionally, it improves load balancing across nodes. A generalization evaluation is conducted across different batch job sizes (200, 500, and 1000 tasks) and heterogeneous cluster configurations (with 1 to 3 types of GPUs), revealing that the proposed method consistently outperforms traditional resource allocation algorithms (Random, RR, SJF) in overall performance, demonstrating strong generalization capability and robustness. Compared to single-objective optimization strategies, the proposed multi-objective optimization model achieves an average improvement of 9.52% in resource utilization, a 131.78% reduction in average job waiting time, and a 16.66% reduction in load imbalance variance, effectively optimizing both scheduling efficiency and fairness in heterogeneous cluster environments. In terms of computational overhead, the DQN-based deep reinforcement learning algorithm not only requires lower training costs but also offers faster decision-making, outperforming common deep reinforcement learning approaches such as DDQN, PPO, and A3C.
Although the proposed method exhibits outstanding performance in multi-objective optimization, future research can explore dynamic weight adjustment mechanisms and scalability for large-scale clusters. Potential directions include integrating federated learning to achieve distributed scheduling and supporting preemptive job scheduling scenarios.

Key words: heterogeneous GPU cluster, deep Q-network, batch jobs, multi-objective optimization

摘要： 异构GPU集群为批次作业处理提供了关键算力,然而传统方法在应对作业需求多样性和资源异构动态性方面存在一定局限。为此,本文提出了一种基于多目标优化的异构GPU集群资源分配方法,构建了综合资源利用率、作业等待时间以及集群负载均衡的多目标资源分配优化模型,设计了马尔科夫决策过程框架并结合深度Q网络(Deep Q-Network, DQN)进行求解。同时,通过构建深度神经网络模型并引入双参数阈值线性衰减策略,优化了模型的训练和决策过程。基于阿里云数据集开展实验验证,结果表明,基于DQN的多目标资源分配模型能够同时有效地优化GPU和CPU利用率、减少批次作业平均等待时间并改善集群负载均衡度;在不同批次作业规模及异构集群配置场景下,相较于传统算法显示出更好的适应性;同时在时间开销方面优于其他深度强化学习方法。本文揭示了深度强化学习算法在异构GPU集群批次作业资源分配中的应用潜力,为异构集群资源管理提供了实践基础。

关键词: 异构GPU集群, 深度Q网络, 批次作业, 多目标优化