关键词: 稀疏性, L₀范数, 支持向量机, 强可转化非凸函数

Abstract: With the widespread application of machine learning classification algorithms in multimodal big data, the accurate classification of high-dimensional data becomes urgent and essential. As the feature dimensions of the classification objects continue to increase, the number of features involved in the final classification result also increases, leading to a decrease in classification accuracy. In practical applications, such high-dimensional classification results need to be more effective. Therefore, in multimodal large models, how to sparsify high-dimensional features has become an urgent issue in many practical classification applications.
Traditional support vector machine models lack feature selection capabilities and are often affected by redundant features, decreasing classification accuracy. Thus, methods of achieving feature sparsification have become crucial. Many researchers have proposed to use methods that involve adding regularization terms for sparsification. Since the L₀ norm is non-convex and non-smooth, belonging to an NP-hard problem, solving it directly is computationally challenging. As a result, some researchers have suggested using the L₁ norm and the L_p norm as penalty terms to simplify the calculations while achieving similar sparsity effects. However, the core idea of these methods is to construct a function that approximates the L₀ norm. Although they address the computational difficulties, there is still room for improvement regarding sparsity.
This paper presents a sparse support vector machine approach based on the L₀ norm. Considering the non-convex and discontinuous nature of the L₀ norm, the paper employs a strongly transformable non-convex function to convert the L₀ norm into a differentiable convex-concave continuous function. For the transformed convex-concave minimax problem, it is equivalent to a bilevel programming problem. The upper-level problem is solved by using both the conjugate gradient descent and the steepest descent algorithm, while the lower-level problem’s optimal solution can be directly obtained and substituted for the upper-level problem. This transformation turns the original problem into a convex optimization problem, thus addressing the difficulty of directly computing the L₀ norm. Due to its equivalence, this model effectively retains the sparsity of the L₀ norm. Finally, a feature block decomposition algorithm named CGDL-SVM is constructed. The basic idea is to divide samples into multiple small blocks based on features and solve them sparsely, and then merge the samples after block sparsification optimization and perform further sparsification optimization to obtain the final decision classification surface. This process simultaneously ensures classification accuracy while reducing the complexity of high-dimensional feature computation.
In the numerical experiments, we first compare the CGDL-SVM algorithm with three sparse support vector machine algorithms using other regularization terms on five UCI datasets, demonstrating that L₀ norm regularization is superior to other regularization terms in terms of sparsity. Then, the CGDL-SVM algorithm is compared with four classical sparse algorithms in terms of sparsity and accuracy on five high-dimensional real-world datasets, and the results show that the CGDL-SVM algorithm not only performs well in terms of classification accuracy, especially excelling in sparsity but also exhibits excellent performance in high-dimensional datasets, indicating good practicality.
In summary, the proposed algorithm in this paper has better sparsity while ensuring high classification accuracy, effectively balancing the contradiction between classification accuracy and sparsity, and providing new ideas for sparse support vector machine research.

Key words: sparse, L₀ norm, support vector machine, strongly convertible nonconvex function