| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
Letter |
kreutz{at}ece.ucsd.edu, Electrical and Computer Engineering, Jacobs School of Engineering, University of California, San Diego, La Jolla, California 92093-0407, U.S.A.
jfmurray{at}ucsd.edu, Electrical and Computer Engineering, Jacobs School of Engineering, University of California, San Diego, La Jolla, California 92093-0407, U.S.A.
brao{at}ece.ucsd.edu, Electrical and Computer Engineering, Jacobs School of Engineering, University of California, San Diego, La Jolla, California 92093-0407, U.S.A.
kjersti.engan{at}tn.his.no, Stavanger University College, School of Science and Technology Ullandhaug, N-4091 Stavanger, Norway
tewon{at}salk.edu, Howard Hughes Medical Institute, Computational Neurobiology Laboratory, Salk Institute, La Jolla, California 92037, U.S.A.
terry{at}salk.edu, Howard Hughes Medical Institute, Computational Neurobiology Laboratory, Salk Institute, La Jolla, California 92037, U.S.A.
Algorithms for data-driven learning of domain-specific overcomplete dictionaries are developed to obtain maximum likelihood and maximum a posteriori dictionary estimates based on the use of Bayesian models with concave/Schur-concave (CSC) negative log priors. Such priors are appropriate for obtaining sparse representations of environmental signals within an appropriately chosen (environmentally matched) dictionary. The elements of the dictionary can be interpreted as concepts, features, or words capable of succinct expression of events encountered in the environment (the source of the measured signals). This is a generalization of vector quantization in that one is interested in a description involving a few dictionary entries (the proverbial "25 words or less"), but not necessarily as succinct as one entry. To learn an environmentally adapted dictionary capable of concise expression of signals generated by the environment, we develop algorithms that iterate between a representative set of sparse representations found by variants of FOCUSS and an update of the dictionary using these sparse representations.
Experiments were performed using synthetic data and natural images. For complete dictionaries, we demonstrate that our algorithms have improved performance over other independent component analysis (ICA) methods, measured in terms of signal-to-noise ratios of separated sources. In the overcomplete case, we show that the true underlying dictionary and sparse sources can be accurately recovered. In tests with natural images, learned overcomplete dictionaries are shown to have higher coding efficiency than complete dictionaries; that is, images encoded with an overcomplete dictionary have both higher compression (fewer bits per pixel) and higher accuracy (lower mean square error).
This article has been cited by other articles:
|
|
 |
|
  C. J. Rozell, D. H. Johnson, R. G. Baraniuk, and B. A. Olshausen Sparse Coding via Thresholding and Local Competition in Neural Circuits Neural Comput., October 1, 2008; 20(10): 2526 - 2563. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
J. F. Murray and K. Kreutz-Delgado Visual recognition and inference using dynamic overcomplete sparse learning. Neural Comput., September 1, 2007; 19(9): 2301 - 2352. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 H. Asari, B. A. Pearlmutter, and A. M. Zador Sparse representations for the cocktail party problem. J. Neurosci., July 12, 2006; 26(28): 7477 - 7490. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
Y. Li, A. Cichocki, and S.-i. Amari Analysis of Sparse Representation and Blind Source Separation Neural Comput., June 1, 2004; 16(6): 1193 - 1234. [Abstract] [Full Text] [PDF]
|
|
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
| J COGNITIVE NEUROSCIENCE | NEURAL COMPUTATION | MIT PRESS JOURNALS |
