Abstract: To meet the huge demand of information industry for computing capacity, China has initiated a project to channel computing resources from the east to the west, which enables large-scare computing capacity scheduling. The computing capacity service platform is the key part of the project which is responsible for providing computing capacity to support massive concurrent services. It is faced with the challenge of heavy processing burden but limited computing capacity resources for delay-sensitive services. In this paper, we aim to study the innovation in deep reinforcement learning based computing capacity service platform. The significance of the research can be summarized as the following three points. Firstly, this paper is the first academic paper to study the core technology of computing capacity service platform. Starting from the challenges and practical background faced by the computing capacity service platform, the paper explores its inherent complex system scheduling and management problems. Secondly, a deep reinforcement learning based computing capacity resource providing strategy is designed for computing capacity service platforms. Thirdly, the performance of the provided strategies is verified through simulations. Countermeasures and suggestions for the subsequent design and construction of the computing capacity service platform are proposed based on simulation results.
The deep reinforcement learning based computing capacity providing strategy is also designed for service platform management. Firstly, a Hidden Markov Model based model for computing capacity chain is proposed considering energy consumption, delay and bandwidth as a multi-objective. The model is proposed mainly for delay sensitive services and considers the processing capacity limitations. Multiple sub-tasks of a computing task are deployed on the computing capacity chain composed of multiple edge network nodes according to networkenvironment changes based on a certain scheduling strategy. Meanwhile, indicators such as energy consumption, delay and bandwidth are considered to optimize the selection of computing capacity providing nodes and the path of computing capacity chain collaboratively. Secondly, the improved list Viterbi based prioritized replay double deep Q network algorithm (VPDDQN) is designed which considers the complex and changing network environment and the huge scheduling action space. VPDDQN is mainly composed of two steps. (1)The improved list Viterbi algorithm selects several optimal candidate scheduling solutions corresponding to a certain state in order to accelerate model training speed and reduce model training difficulty. (2)DDQN, which is a kind of deep reinforcement learning algorithms, is improved to select the best scheduling solution of the candidate solutions. In this way, the proposed strategy can make the optimal scheduling and optimization solution for edge computing capacity chain according the state of network environment.
Taking the project to channel computing resources from the east to the west as a case background, we select two cities in Beijing-Tianjin-Hebei region as the simulation scenario. The simulation results are as follows. Firstly, the proposed algorithm VPDDQN, which is used as computing capacity providing strategy for the project, is efficient and the model training time is short for VPDDQN. Secondly, the proposed algorithm has the best overall performance of the other benchmark algorithms in terms of delay, bandwidth and energy consumption. Thirdly, the proposed computing capacity providing strategy can achieve the maximum completion rate of computing tasks for different scales of computing tasks, thereby improving the economic efficiency of computing capacity network operation. The simulation results show that the proposed strategy can help the platform improve the performance and economy of computing capacity provision.
Finally, we provide policy implications and suggestions for building the computing capacity service platform from the perspectives of intercity computing capacity collaboration, information collection and regional tasks planning. Firstly, reasonable and efficient computing capacity providing strategies have a crucial positive impact on the construction and operation of the project. Secondly, the management department needs to recognize the complexity of the multi-point collaborative computing capacity chain providing method and collect real-time information about the delay, bandwidth and energy consumption of geographically distributed edge computing sites. Thirdly, task plans should be made according to local conditions to improve completion rate of computing tasks and the economy of computing power network operation.

Key words: computing capacity service platform, channel computing resources from the east to the west, deep reinforcement learning

摘要： 算力服务平台负责为海量并发业务提供算力调度支撑。在算力资源有限的条件下,面临时延敏感业务处理负担重但算力资源有限的挑战。为解决这一问题,提出基于深度强化学习的算力资源供给策略。首先将能耗、时延和带宽占用作为多目标,基于隐马尔科夫模型建立算力链供给模型,但网络环境动态变化以及调度动作空间极大导致难以直接求解。因此,使用改进的列表维特比(Viterbi)算法提供较优解集,进而使用改进的优先级回放双深度Q网络算法进一步求解,从而根据网络环境变化自适应制定供给方案。以中国东数西算重大工程作为案例背景,选取京津冀枢纽中两个城市的算力资源供给作为仿真场景。仿真结果表明,所提策略可帮助算力服务平台提升算力供给性能以及经济性。最后,从城市间算力协作、信息管理和区域任务规划三个方面为算力服务平台建设提供对策建议。

关键词: 算力服务平台, 东数西算, 深度强化学习