Neural Computation, Vol 8, 1541-1565, Copyright © 1996 by The MIT Press

ARTICLES

Hybrid modeling, HMM/NN architectures, and protein applications

P Baldi and Y Chauvin
Division of Biology, California Institute of Technology, Pasadena 91125, USA.

We describe a hybrid modeling approach where the parameters of a mode are calculated and modulated by another model, typically a neural network (NN), to avoid both overfitting and underfitting. We develop the approach for the case of Hidden Markov Models (HMMs), by deriving a class of hybrid HMM/NN architectures. These architectures can be trained with unified algorithms that blend HMM dynamic programming with NN backpropagation. In the case of complex data, mixtures of HMMs or modulated HMMs must be used. NNs can then be applied both to the parameters of each single HMM, and to the switching or modulatation of the models, as a function of input or context. Hybrid HMM/NN architectures provide a flexible NN parameterization for the control of model structure and complexity. At the same time, they can capture distributions that, in practice, are inaccessible to single HMMs. The HMM/NN hybrid approach is tested, in its simplest form, by constructing a model of the immunoglobulin protein family. A hybrid model is trained, and a multiple alignment derived, with less than a fourth of the number of parameters used with previous single HMMs.

This article has been cited by other articles:

				J.-P. Z. Wang and J. Widom Improved alignment of nucleosome DNA sequences using a mixture model Nucleic Acids Res., December 9, 2005; 33(21): 6743 - 6755. [Abstract] [Full Text] [PDF]