Browse all publications by topic
Browse all publications by year
- C. Weber and
K. Obermayer. Structured models from structured data: emergence of modular
information processing within one sheet of neurons.
.
In Neural Networks - IJCNN 2000, volume IV, pages 608-613, 2000.
(FTP Gzipped PostScript, 6 pages, 43 kb)
In our contribution we investigate how structured information
processing within a neural net can emerge as a result of unsupervised
learning from data. Our model consists of input neurons and hidden neurons
which are recurrently connected and which represent the thalamus and the
cortex, respectively. On the basis of a maximum likelihood framework the task
is to generate given input data using the code of the hidden units. Hidden
neurons are fully connected allowing for different roles to play within the
unfolding time-dynamics of this data generation process. One parameter which
is related to the sparsity of neuronal activation varies across the hidden
neurons. As a result of training the net captures the structure of the data
generation process. Trained on data which are generated by different
mechanisms acting in parallel, the more active neurons will code for the more
frequent input features. Trained on hierarchically generated data, the more
active neurons will code on the higher level where each feature integrates
several lower level features. The results imply that the division of the
cortex into laterally and hierarchically organized areas can evolve to a
certain degree as an adaptation to the environment.
|
