HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS		QUICK SEARCH:   [advanced] Author: Keyword(s): Year:  Vol:  Page:

This Article Full Text Full Text (PDF) Alert me when this article is cited Alert me if a correction is posted Services Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Vijayakumar, S. Articles by Schaal, S. Search for Related Content PubMed PubMed Citation Articles by Vijayakumar, S. Articles by Schaal, S.

Incremental Online Learning in High Dimensions

sethu.vijayakumar{at}ed.ac.uk, School of Informatics, University of Edinburgh, Edinburgh EH9 3JZ, U.K.

Aaron D'Souza

adsouza{at}usc.edu, Department of Computer Science, University of Southern California, Los Angeles, CA 90089-2520, U.S.A.

Stefan Schaal

sschaal{at}usc.edu, Department of Computer Science, University of Southern California, Los Angeles, CA 90089-2520, U.S.A.

Locally weighted projection regression (LWPR) is a new algorithm for incremental nonlinear function approximation in high-dimensional spaces with redundant and irrelevant input dimensions. At its core, it employs nonparametric regression with locally linear models. In order to stay computationally efficient and numerically robust, each local model performs the regression analysis with a small number of univariate regressions in selected directions in input space in the spirit of partial least squares regression. We discuss when and how local learning techniques can successfully work in high-dimensional spaces and review the various techniques for local dimensionality reduction before finally deriving the LWPR algorithm. The properties of LWPR are that it (1) learns rapidly with second-order learning methods based on incremental training, (2) uses statistically sound stochastic leave-one-out cross validation for learning without the need to memorize training data, (3) adjusts its weighting kernels based on only local information in order to minimize the danger of negative interference of incremental learning, (4) has a computational complexity that is linear in the number of inputs, and (5) can deal with a large number of—possibly redundant—inputs, as shown in various empirical evaluations with up to 90 dimensional data sets. For a probabilistic interpretation, predictive variance and confidence intervals are derived. To our knowledge, LWPR is the first truly incremental spatially localized learning method that can successfully and efficiently operate in very high-dimensional spaces.

This article has been cited by other articles:

				J. Peters and S. Schaal Learning to Control in Operational Space The International Journal of Robotics Research, February 1, 2008; 27(2): 197 - 212. [Abstract] [PDF]