HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS		QUICK SEARCH:   [advanced] Author: Keyword(s): Year:  Vol:  Page:

This Article Full Text Full Text (PDF) Alert me when this article is cited Alert me if a correction is posted Services Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Amari, S.-i. Articles by Fukumizu, K. Search for Related Content PubMed PubMed Citation Articles by Amari, S.-i. Articles by Fukumizu, K.

Adaptive Method of Realizing Natural Gradient Learning for Multilayer Perceptrons

RIKEN Brain Science Institute, Wako-shi, Hirosawa, Saitama 351-0198, Japan

Hyeyoung Park

RIKEN Brain Science Institute, Wako-shi, Hirosawa, Saitama 351-0198, Japan

Kenji Fukumizu

RIKEN Brain Science Institute, Wako-shi, Hirosawa, Saitama 351-0198, Japan

The natural gradient learning method is known to have ideal performances for on-line training of multilayer perceptrons. It avoids plateaus, which give rise to slow convergence of the backpropagation method. It is Fisher efficient, whereas the conventional method is not. However, for implementing the method, it is necessary to calculate the Fisher information matrix and its inverse, which is practically very difficult. This article proposes an adaptive method of directly obtaining the inverse of the Fisher information matrix. It generalizes the adaptive Gauss-Newton algorithms and provides a solid theoretical justification of them. Simulations show that the proposed adaptive method works very well for realizing natural gradient learning.

This article has been cited by other articles:

				S.-i. Amari, H. Park, and T. Ozeki Singularities affect dynamics of learning in neuromanifolds. Neural Comput., May 1, 2006; 18(5): 1007 - 1065. [Abstract] [Full Text] [PDF]

				J. Basak Online Adaptive Decision Trees Neural Comput., September 1, 2004; 16(9): 1959 - 1981. [Abstract] [Full Text] [PDF]

				H. Park, N. Murata, and S.-i. Amari Improving Generalization Performance of Natural Gradient Learning Using Optimized Regularization by NIC Neural Comput., February 1, 2004; 16(2): 355 - 382. [Abstract] [Full Text] [PDF]