Browse all publications by topic
Browse all publications by year
- R. Vollgraf and
K. Obermayer. Sparse Optimization for Second Order Kernel Methods.
.
In IJCNN 2006 Conference Proceedings, pages 145-152, 2006.
(FTP PDF, 1188 kb)
We present a new optimization procedure which is particularly
suited for the solution of second-order kernel methods like e.g. Kernel-PCA.
Common to these methods is that there is a cost function to be optimized,
under a positive definite quadratic constraint, which bounds the solution.
For example, in Kernel-PCA the constraint provides unit length and orthogonal
(in feature space) principal components. The cost function is often quadratic
which allows to solve the problem as a generalized eigenvalue problem.
However, in contrast to Support Vector Machines, which employ box
constraints, quadratic constraints usually do not lead to sparse solutions.
Here we give up the structure of the generalized eigenvalue problem in favor
of a non-quadratic regularization term added to the cost function, which
enforces sparse solutions. To optimize this more ``complicated'' cost
function, we introduce a modified conjugate gradient descent method. Starting
from an admissible point, all iterations are carried out inside the subspace
of admissible solutions, which is defined by the hyper-ellipsoidal constraint
surface.
|
