Abstract
Vector quantization, a central topic in data compression, deals with the problem of encoding an information source or a sample of data vectors by means of a finite codebook, such that the average distortion is minimized. We introduce a common framework, based on maximum entropy inference to derive a deterministic annealing algorithm for robust vector quantization. The objective function for codebook design is extended to take channel noise and bandwidth limitations into account. Formulated as an on-line problem it is possible to derive learning rules for competitive neural networks. The resulting update rule is a generalization of the ‘neural gas’ model. The foundation in coding theory allows us to specify an optimality criterion for the ‘neural gas’ update rule.
Supported by the Federal Ministry for Education, Science and Technology (BMBF) under grant #01 M 3021 A/4
This is a preview of subscription content, log in via an institution to check access.
Preview
Unable to display preview. Download preview PDF.
References
J. M. Buhmann and H. Kühnel. Complexity optimized data clustering by competitive neural networks. Neural Computation, 5:75–88, 1993.
J. M. Buhmann and H. Kühnel. Vector quantization with complexity costs. IEEE Transactions on Information Theory, 39(4): 1133–1145, July 1993.
A. Gersho and R. M. Gray. Vector Quantization and Signal Processing. Kluwer Academic Publisher, Boston, 1992.
T. Kohonen. Self-organization and Associative Memory. Springer, Berlin, 1984.
J.J. Kosowsky and A.L. Yuille. The invisible hand algorithm: solving the assignment problem with statistical mechanics. Neural Computation, 7(3):477–490, 1994.
Y. Linde, A. Buzo, and R. M. Gray. An algorithm for vector quantizer design. IEEE Transactions on Communications, 28:84–95, 1980.
S.P. Luttrell. Hierarchical vector quantizations. IEE Proceedings, 136:405–413, 1989.
J. MacQueen. Some methods for classification and analysis of multivariate observations. In Proceedings of the 5th Berkeley Symposium on Mathematical Statistics and Probability, pages 281–297, 1967.
S. Mallat. A theory for multidimensional signal decomposition: the wavelet representation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 11 (7):674–693, 1989.
T.M. Martinetz, S. G. Berkovich, and K. J. Schulten, 'Neural-gas’ network for vector quantization and its application to time-series prediction. IEEE Transactions on Neural Networks 4(4):558–569, 1993.
K. Rose, E. Gurewitz, and G. Fox. Statistical mechanics and phase transitions in clustering. Physical Review Letters, 65(8):945–948, 1990.
R. Sinkhorn. A relationship between arbitrary positive matrices and doubly stochastic matrices. Ann. Math. Statist., 35:876–879, 1964.
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 1996 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Hofmann, T., Buhmann, J.M. (1996). An annealed ‘neural gas’ network for robust vector quantization. In: von der Malsburg, C., von Seelen, W., Vorbrüggen, J.C., Sendhoff, B. (eds) Artificial Neural Networks — ICANN 96. ICANN 1996. Lecture Notes in Computer Science, vol 1112. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-61510-5_29
Download citation
DOI: https://doi.org/10.1007/3-540-61510-5_29
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-61510-1
Online ISBN: 978-3-540-68684-2
eBook Packages: Springer Book Archive