HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS		QUICK SEARCH:   [advanced] Author: Keyword(s): Year:  Vol:  Page:

This Article Full Text Full Text (PDF) Alert me when this article is cited Alert me if a correction is posted Services Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Højen-Sørensen, P. A.d.F.R. Articles by Hansen, L. K. Search for Related Content PubMed PubMed Citation Articles by Højen-Sørensen, P. A.d.F.R. Articles by Hansen, L. K.

Mean-Field Approaches to Independent Component Analysis

phs{at}imm.dtu.dk, Department of Mathematical Modelling, Technical University of Denmark, DK-2800 Lyngby, Denmark

Ole Winther

winther{at}cbs.dtu.dk, Department of Mathematical Modelling and Center for Biological Sequence Analysis, Department of Biotechnology, Technical University of Denmark, DK-2800 Lyngby, Denmark

Lars Kai Hansen

lkhansen{at}imm.dtu.dk, Department of Mathematical Modelling, Technical University of Denmark, DK-2800 Lyngby, Denmark

We develop mean-field approaches for probabilistic independent component analysis (ICA). The sources are estimated from the mean of their posterior distribution and the mixing matrix (and noise level) is estimated by maximum a posteriori (MAP). The latter requires the computation of (a good approximation to) the correlations between sources. For this purpose, we investigate three increasingly advanced mean-field methods: the variational (also known as naive mean field) approach, linear response corrections, and an adaptive version of the Thouless, Anderson and Palmer (1977) (TAP) mean-field approach, which is due to Opper and Winther (2001). The resulting algorithms are tested on a number of problems. On synthetic data, the advanced mean-field approaches are able to recover the correct mixing matrix in cases where the variational mean-field theory fails. For handwritten digits, sparse encoding is achieved using nonnegative source and mixing priors. For speech, the mean-field method is able to separate in the underdetermined (overcomplete) case of two sensors and three sources. One major advantage of the proposed method is its generality and algorithmic simplicity. Finally, we point out several possible extensions of the approaches developed here.

This article has been cited by other articles:

				R. K. Olsson, K. B. Petersen, and T. Lehn-Schioler State-space models: from the em algorithm to a gradient approach. Neural Comput., April 1, 2007; 19(4): 1097 - 1111. [Abstract] [Full Text] [PDF]

				M. C. Tresch, V. C. K. Cheung, and A. d'Avella Matrix Factorization Algorithms for the Identification of Muscle Synergies: Evaluation on Simulated and Experimental Data Sets J Neurophysiol, April 1, 2006; 95(4): 2199 - 2212. [Abstract] [Full Text] [PDF]

				K. B. Petersen, O. Winther, and L. K. Hansen On the Slow Convergence of EM and VBEM in Low-Noise Linear Models Neural Comput., September 1, 2005; 17(9): 1921 - 1926. [Abstract] [Full Text] [PDF]

				S.-i. Maeda, W.-J. Song, and S. Ishii Nonlinear and Noisy Extension of Independent Component Analysis: Theory and Its Application to a Pitch Sensation Model Neural Comput., January 1, 2005; 17(1): 115 - 144. [Abstract] [Full Text] [PDF]