Browse all publications by topic
Browse all publications by year
- T. Graepel, M. Burger, and
K. Obermayer. Deterministic Annealing for Topographic Vector Quantization
and Self-Organizing Maps.
.
In T. Kohonen, editor, Proceedings of the Workshop on Self-Organizing
Maps - WSOM 97, pages 345-350, 1997.
(FTP Gzipped PostScript, 6 pages, 101 kb)
We have developed a robust optimization scheme for self-organizing
maps in the framework of noisy vector quantization. Based on a cost function
that takes distortions from channel noise into account we derive a fuzzy
algorithm of EM-type for topographic vector quantization (STVQ) which employs
deterministic annealing. This annealing process leads to phase transitions in
the cluster representation for which we are able to calculate critical modes
and temperatures as a function of the neighbourhood function and the
covariance matrix of the data. Similar results are obtained for the automatic
selection of feature dimensions. Deterministic annealing also offers an
alternative to the heuristic stepwise shrinking of the neighbourhood width in
the SOM and makes it possible to use the neighbourhood solely to encode
desired neighbourhood relations between the clusters. A soft version of the
SOM (SSOM) is derived as a computationally efficient approximation to the
E-step of STVQ. Both methods are numerically tested on a two-dimensional map
of the plane and we conclude that the temperature annealing can be precisely
controlled and could for many applications be the method of choice. We have
developed a robust optimization scheme for self-organizing maps in the
framework of noisy vector quantization. Based on a cost function that takes
distortions from channel noise into account we derive a fuzzy algorithm of
EM-type for topographic vector quantization (STVQ) which employs
deterministic annealing. This annealing process leads to phase transitions in
the cluster representation for which we are able to calculate critical modes
and temperatures as a function of the neighbourhood function and the
covariance matrix of the data. Similar results are obtained for the automatic
selection of feature dimensions. Deterministic annealing also offers an
alternative to the heuristic stepwise shrinking of the neighbourhood width in
the SOM and makes it possible to use the neighbourhood solely to encode
desired neighbourhood relations between the clusters. A soft version of the
SOM (SSOM) is derived as a computationally efficient approximation to the
E-step of STVQ. Both methods are numerically tested on a two-dimensional map
of the plane and we conclude that the temperature annealing can be precisely
controlled and could for many applications be the method of
choice.
|
