HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS		QUICK SEARCH:   [advanced] Author: Keyword(s): Year:  Vol:  Page:

This Article Full Text Full Text (PDF) Alert me when this article is cited Alert me if a correction is posted Services Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Karklin, Y. Articles by Lewicki, M. S. Search for Related Content PubMed PubMed Citation Articles by Karklin, Y. Articles by Lewicki, M. S.

A Hierarchical Bayesian Model for Learning Nonlinear Statistical Regularities in Nonstationary Natural Signals

yan+{at}cs.cmu.edu, Computer Science Department and Center for the Neural Basis of Cognition, Carnegie Mellon University, Pittsburgh, PA 15213, U.S.A.

Michael S. Lewicki

lewicki{at}cnbc.cmu.edu, Computer Science Department and Center for the Neural Basis of Cognition, Carnegie Mellon University, Pittsburgh, PA 15213, U.S.A.

Capturing statistical regularities in complex, high-dimensional data is an important problem in machine learning and signal processing. Models such as principal component analysis (PCA) and independent component analysis (ICA) make few assumptions about the structure in the data and have good scaling properties, but they are limited to representing linear statistical regularities and assume that the distribution of the data is stationary. For many natural, complex signals, the latent variables often exhibit residual dependencies as well as nonstationary statistics. Here we present a hierarchical Bayesian model that is able to capture higher-order nonlinear structure and represent nonstationary data distributions. The model is a generalization of ICA in which the basis function coefficients are no longer assumed to be independent; instead, the dependencies in their magnitudes are captured by a set of density components. Each density component describes a common pattern of deviation from the marginal density of the pattern ensemble; in different combinations, they can describe nonstationary distributions. Adapting the model to image or audio data yields a nonlinear, distributed code for higher-order statistical regularities that reflect more abstract, invariant properties of the signal.

This article has been cited by other articles:

				R. Turner and M. Sahani A maximum-likelihood interpretation for slow feature analysis. Neural Comput., April 1, 2007; 19(4): 1022 - 1038. [Abstract] [Full Text] [PDF]

				F. Creutzig and H. Sprekeler Predictive Coding and the Slowness Principle: An Information-Theoretic Approach Neural Comput., April 1, 2007; 20(4): 1026 - 1041. [Abstract] [Full Text] [PDF]

				O. Schwartz, T. J. Sejnowski, and P. Dayan Soft mixer assignment in a hierarchical generative model of natural scene statistics. Neural Comput., November 1, 2006; 18(11): 2680 - 2718. [Abstract] [Full Text] [PDF]

				S. Osindero, M. Welling, and G. E. Hinton Topographic Product Models Applied to Natural Scene Statistics Neural Comput., February 1, 2005; 18(2): 381 - 414. [Abstract] [Full Text] [PDF]