Neural Comp. NEW Faster Access
HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS
QUICK SEARCH:   [advanced]


     

This Article
Right arrow Full Text (PDF)
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Services
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Right arrow reprints & permissions
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Roweis, S.
Right arrow Articles by Ghahramani, Z.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Roweis, S.
Right arrow Articles by Ghahramani, Z.

Neural Computation, Vol 11, Issue 2 305-345, Copyright © 1999 by The MIT Press

REVIEWS

A unifying review of linear gaussian models

S Roweis and Z Ghahramani
Department of Molecular Biology, Princeton University, Princeton, NJ 08544, USA. roweis@gatsby.ucl.ac.uk

Factor analysis, principal component analysis, mixtures of gaussian clusters, vector quantization, Kalman filter models, and hidden Markov models can all be unified as variations of unsupervised learning under a single basic generative model. This is achieved by collecting together disparate observations and derivations made by many previous authors and introducing a new way of linking discrete and continuous state models using a simple nonlinearity. Through the use of other nonlinearities, we show how independent component analysis is also a variation of the same basic generative model. We show that factor analysis and mixtures of gaussians can be implemented in autoencoder neural networks and learned using squared error plus the same regularization term. We introduce a new model for static data, known as sensible principal component analysis, as well as a novel concept of spatially adaptive observation noise. We also review some of the literature involving global and local mixtures of the basic models and provide pseudocode for inference and learning for all the basic models.


This article has been cited by other articles:


Home page
 Neural Comput.Home page
R. Turner and M. Sahani
A maximum-likelihood interpretation for slow feature analysis.
Neural Comput., April 1, 2007; 19(4): 1022 - 1038.
[Abstract] [Full Text] [PDF]

Home page
 Neural Comput.Home page
R. K. Olsson, K. B. Petersen, and T. Lehn-Schioler
State-space models: from the em algorithm to a gradient approach.
Neural Comput., April 1, 2007; 19(4): 1097 - 1111.
[Abstract] [Full Text] [PDF]

Home page
 Biophys. JHome page
L. S. Milescu, A. Yildiz, P. R. Selvin, and F. Sachs
Extracting Dwell Time Sequences from Processive Molecular Motor Data
Biophys. J., November 1, 2006; 91(9): 3135 - 3150.
[Abstract] [Full Text] [PDF]

Home page
 J. Neurophysiol.Home page
M. C. Tresch, V. C. K. Cheung, and A. d'Avella
Matrix Factorization Algorithms for the Identification of Muscle Synergies: Evaluation on Simulated and Experimental Data Sets
J Neurophysiol, April 1, 2006; 95(4): 2199 - 2212.
[Abstract] [Full Text] [PDF]

Home page
 Behav Cogn Neurosci RevHome page
W. A. Suzuki and E. N. Brown
Behavioral and Neurophysiological Analyses of Dynamic Learning Processes
Behav Cogn Neurosci Rev, June 1, 2005; 4(2): 67 - 95.
[Abstract] [PDF]

Home page
 J. Neurophysiol.Home page
A. C. Smith, M. R. Stefani, B. Moghaddam, and E. N. Brown
Analysis and Design of Behavioral Experiments to Characterize Population Learning
J Neurophysiol, March 1, 2005; 93(3): 1776 - 1792.
[Abstract] [Full Text] [PDF]

Home page
 BioinformaticsHome page
M. J. Beal, F. Falciani, Z. Ghahramani, C. Rangel, and D. L. Wild
A Bayesian approach to reconstructing genetic regulatory networks with hidden factors
Bioinformatics, February 1, 2005; 21(3): 349 - 356.
[Abstract] [Full Text] [PDF]

Home page
 Neural Comput.Home page
E. Lopez-Rubio, J. M. Ortiz-de-Lazcano-Lobato, J. Munoz-Perez, and J. Antonio Gomez-Ruiz
Principal Components Analysis Competitive Learning
Neural Comput., November 1, 2004; 16(11): 2459 - 2481.
[Abstract] [Full Text] [PDF]

Home page
 J. Neurophysiol.Home page
B. B. Averbeck and L. M. Romanski
Principal and Independent Components of Macaque Vocalizations: Constructing Stimuli to Probe High-Level Sensory Processing
J Neurophysiol, June 1, 2004; 91(6): 2897 - 2909.
[Abstract] [Full Text] [PDF]

Home page
 J. Neurosci.Home page
A. C. Smith, L. M. Frank, S. Wirth, M. Yanike, D. Hu, Y. Kubota, A. M. Graybiel, W. A. Suzuki, and E. N. Brown
Dynamic Analysis of Learning in Behavioral Experiments
J. Neurosci., January 14, 2004; 24(2): 447 - 461.
[Abstract] [Full Text] [PDF]

Home page
 Neural Comput.Home page
G. de A. Barreto, A. F. R. Araujo, and S. C. Kremer
A Taxonomy for Spatiotemporal Connectionist Networks Revisited: The Unsupervised Case
Neural Comput., June 1, 2003; 15(6): 1255 - 1320.
[Abstract] [Full Text] [PDF]

Home page
 Neural Comput.Home page
A. C. Smith and E. N. Brown
Estimating a State-Space Model from Point Process Observations
Neural Comput., May 1, 2003; 15(5): 965 - 991.
[Abstract] [Full Text] [PDF]

Home page
 Neural Comput.Home page
M.-a. Sato
Online Model Selection Based on the Variational Bayes
Neural Comput., July 1, 2001; 13(7): 1649 - 1681.
[Abstract] [Full Text]

Home page
 Neural Comput.Home page
R. Rosipal and M. Girolami
An Expectation-Maximization Approach to Nonlinear Component Analysis
Neural Comput., March 1, 2001; 13(3): 505 - 510.
[Abstract] [Full Text]

Home page
 Neural Comput.Home page
P. Meinicke and H. Ritter
Resolution-Based Complexity Control for Gaussian Mixture Models
Neural Comput., February 1, 2001; 13(2): 453 - 475.
[Abstract] [Full Text]

Home page
 Neural Comput.Home page
Z. Ghahramani and G. E. Hinton
Variational Learning for Switching State-Space Models
Neural Comput., April 1, 2000; 12(4): 831 - 864.
[Abstract] [Full Text]


HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS
J COGNITIVE NEUROSCIENCE NEURAL COMPUTATION MIT PRESS JOURNALS
Copyright © 1999 by The MIT Press.