关键词: 无约束优化, 共轭梯度法, 强收敛, 图像恢复, 交通配流

Abstract: Unconstrained optimization serves as the foundational basis for investigating constrained optimization problems. Moreover, numerous practical problems in fields including economics, management, engineering control, and artificial intelligence can be directly or indirectly modeled as unconstrained optimization models. Commonly used methods for solving unconstrained optimization problems include the Newton method, quasi-Newton methods, and the conjugate gradient method. Among these approaches, the conjugate gradient method eliminates the need to compute the Hessian matrix of the objective function or the inverse of its approximate matrix in each iteration. This characteristic endows the conjugate gradient method with advantages such as simple iteration steps and low memory consumption, making it one of the most competitive algorithms for solving large-scale unconstrained optimization problems.
The conjugate gradient methods include the classical conjugate gradient methods Hestenes-Stiefel (HS) method, Fletcher-Reeves (FR) method, Polak-Ribière-Polyak (PRP) method, Conjugate-Descent (CD) method, Liu-Storey (LS) method and Dai-Yuan (DY) method and their improved variants. In this paper, motivated by the single-parameter conjugate parameter proposed by DAI and YUAN (2003), an efficient conjugate gradient method is proposed by introducing a novel conjugate parameter strategy and a resrart procedure. Here, the conjugate parameter is derived from a two-parameter conjugate gradient method family generated by FR, CD, and DY. To ensure the algorithm achieves strong convergence, the denominator of the conjugate parameter is further truncated. Moreover, to enhance the computational efficiency of the algorithm, a resrart procedure related to the conjugate parameter is incorporated into the design of the search direction to accelerate the algorithm’s descent. Under conventional assumptions and the standard Wolfe line search, the strong convergence of the algorithm is established. We aim to develop an efficient conjugate gradient method for solving large-scale unconstrained optimization problems, thereby providing an effective algorithmic model for practical applications such as image restoration, machine learning, and traffic assignment.
Two notable theoretical advantages of the proposed algorithm are as follows: its generated search direction always satisfies the sufficient descent condition, independent of any specific line search strategy; and under conventional assumptions such as the Lipschitz continuity of the gradient and the boundedness below the level sets, the algorithm is proven to possess strong convergence when combined with a standard Wolfe line search for determining the step size.
To evaluate the practical performance of the proposed algorithm, this paper conducts three sets of numerical experiments. Firstly, under the same computational environment, a numerical comparison of six algorithms is carried out on 102 unconstrained optimization test problems, evaluated from three dimensions: computational time, number of iterations, and solution accuracy. The numerical results show that the numerical performance of the proposed algorithm is generally superior to other methods on the test problems. Secondly, the presented algorithm is applied to deal with the image restoration problem, which involves deblurring and denoising blurred and noisy images via a regularized least-squares formulation, which shows that the proposed method is effective on the tested images. Finally, the proposed algorithm to handle the traffic assignment problem in transportation network modeling involves solving a large-scale network equilibrium problem. Our algorithm contributes to achieving a more efficient and realistic flow distribution, and significantly reduces the runtime compared to traditional solvers.
The numerical results indicate that the newly designed algorithm is not only theoretically sound but also highly effective and versatile across different application domains. It provides researchers and practitioners in science and engineering with an effective tool for handling large-scale optimization problems.

Key words: unconstrained optimization, conjugate gradient method, strong convergence, image restoration, traffic assignment