HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS		QUICK SEARCH:   [advanced] Author: Keyword(s): Year:  Vol:  Page:

This Article Full Text Full Text (PDF) Alert me when this article is cited Alert me if a correction is posted Services Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Slonim, N. Articles by Tishby, N. Search for Related Content PubMed PubMed Citation Articles by Slonim, N. Articles by Tishby, N.

Multivariate Information Bottleneck

nslonim{at}princeton.edu Department of Physics and the Lewis–Sigler Institute for Integrative Genomics, Princeton University, Princeton, NJ 08544, U.S.A.

Nir Friedman

nir{at}cs.huji.ac.il School of Computer Science and Engineering, Hebrew University, Jerusalem 91904, Israel

Naftali Tishby

tishby{at}cs.huji.ac.il School of Computer Science and Engineering, and Interdisciplinary Center for Neural Computation, Hebrew University, Jerusalem 91904, Israel

The information bottleneck (IB) method is an unsupervised model independent data organization technique. Given a joint distribution, p(X, Y), this method constructs a new variable, T, that extracts partitions, or clusters, over the values of X that are informative about Y. Algorithms that are motivated by the IB method have already been applied to text classification, gene expression, neural code, and spectral analysis. Here, we introduce a general principled framework for multivariate extensions of the IB method. This allows us to consider multiple systems of data partitions that are interrelated. Our approach utilizes Bayesian networks for specifying the systems of clusters and which information terms should be maintained. We show that this construction provides insights about bottleneck variations and enables us to characterize the solutions of these variations. We also present four different algorithmic approaches that allow us to construct solutions in practice and apply them to several real-world problems.

This article has been cited by other articles:

				F. Creutzig and H. Sprekeler Predictive Coding and the Slowness Principle: An Information-Theoretic Approach Neural Comput., April 1, 2007; 20(4): 1026 - 1041. [Abstract] [Full Text] [PDF]