Browse all publications by topic
Browse all publications by year
- S. Seo and K. Obermayer.
Dynamic Hyperparameter Scaling Method for LVQ Algorithms.
.
In IJCNN 2006 Conference Proceedings, pages 3196-3203, 2006.
(FTP PDF, 222 kb)
We propose a new annealing method for the hyperparameters of
several recent Learning Vector Quantization algorithms. We first analyze the
relationship between values assigned to the hyperparameters, the on-line
learning process, and the structure of the resulting classifier. Motivated by
the results we then suggest an annealing method, where each hyperparameter is
initially set to a large value and is then slowly decreased during learning.
We apply the annealing method to the LVQ 2.1, SLVQ-LR, and RSLVQ methods, and
we compare the generalization performance achieved with the new annealing
method and with a standard hyperparameter selection using 10-fold cross
validation. Benchmark results are provided for the datasets letter and
pendigits from the UCI Machine Learning Repository. The new selection method
provides equally good or - for some data sets - even superior results when
compared to standard selection methods. More importantly, however, the number
of learning trials for different values of the hyperparameters is drastically
reduced. The results are insensitive to the form and parameters of the
annealing schedule.
|
