Neural Computation, Vol 7, 889-904, Copyright © 1995 by The MIT Press

ARTICLES

The Helmholtz machine

P Dayan, GE Hinton, RM Neal and RS Zemel
Department of Computer Science, University of Toronto, Ontario, Canada.

Discovering the structure inherent in a set of patterns is a fundamental aim of statistical inference or learning. One fruitful approach is to build a parameterized stochastic generative model, independent draws from which are likely to produce the patterns. For all but the simplest generative models, each pattern can be generated in exponentially many ways. It is thus intractable to adjust the parameters to maximize the probability of the observed patterns. We describe a way of finessing this combinatorial explosion by maximizing an easily computed lower bound on the probability of the observations. Our method can be viewed as a form of hierarchical self-supervised learning that may relate to the function of bottom-up and top-down cortical processing pathways.

This article has been cited by other articles:

				S.-i. Amari Integration of Stochastic Models by Minimizing {alpha}-Divergence Neural Comput., October 1, 2007; 19(10): 2780 - 2796. [Abstract] [Full Text] [PDF]

				N. V. Swindale Feedback Decoding of Spatially Structured PopulationActivity in Cortical Maps Neural Comput., January 1, 2007; 20(1): 176 - 204. [Abstract] [Full Text] [PDF]

				P. Dayan Images, Frames, and Connectionist Hierarchies. Neural Comput., October 1, 2006; 18(10): 2293 - 2319. [Abstract] [Full Text] [PDF]

				S. Wu and S.-i. Amari Computing with Continuous Attractors: Stability and Online Aspects Neural Comput., October 1, 2005; 17(10): 2215 - 2239. [Abstract] [Full Text] [PDF]

				R. P. N. Rao Bayesian Computation in Recurrent Neural Circuits Neural Comput., January 1, 2004; 16(1): 1 - 38. [Abstract] [Full Text] [PDF]

				D. Graboi and J. Lisman Recognition by Top-Down and Bottom-Up Processing in Cortex: The Control of Selective Attention J Neurophysiol, August 1, 2003; 90(2): 798 - 810. [Abstract] [Full Text] [PDF]

				G. de A. Barreto, A. F. R. Araujo, and S. C. Kremer A Taxonomy for Spatiotemporal Connectionist Networks Revisited: The Unsupervised Case Neural Comput., June 1, 2003; 15(6): 1255 - 1320. [Abstract] [Full Text] [PDF]

				S. Wu, D. Chen, M. Niranjan, and S.-i. Amari Sequential Bayesian Decoding with a Population of Neurons Neural Comput., May 1, 2003; 15(5): 993 - 1012. [Abstract] [Full Text] [PDF]

				J. B. Tenenbaum and W. T. Freeman Separating Style and Content with Bilinear Models Neural Comput., June 1, 2000; 12(6): 1247 - 1283. [Abstract] [Full Text]

				L. K. Saul and M. I. Jordan Attractor Dynamics in Feedforward Neural Networks Neural Comput., June 1, 2000; 12(6): 1313 - 1335. [Abstract] [Full Text]

				H. Attias Independent Factor Analysis Neural Comput., May 15, 1999; 11(4): 803 - 851. [Abstract] [Full Text]

				B. J. Frey and G. E. Hinton Variational Learning in Nonlinear Gaussian Belief Networks Neural Comput., January 15, 1999; 11(1): 193 - 213. [Abstract] [Full Text]