HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS		QUICK SEARCH:   [advanced] Author: Keyword(s): Year:  Vol:  Page:

This Article Full Text Full Text (PDF) Alert me when this article is cited Alert me if a correction is posted Services Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via Google Scholar Google Scholar Articles by Chen, Z. Articles by Haykin, S. Search for Related Content PubMed PubMed Citation Articles by Chen, Z. Articles by Haykin, S.

On Different Facets of Regularization Theory

zhechen{at}soma.crl.mcmaster.ca, Adaptive Systems Lab, Communications Research Laboratory, McMaster University, Hamilton, Ontario, Canada L8S 4K1

Simon Haykin

haykin{at}mcmaster.ca, Adaptive Systems Lab, Communications Research Laboratory, McMaster University, Hamilton, Ontario, Canada L8S 4K1

This review provides a comprehensive understanding of regularization theory from different perspectives, emphasizing smoothness and simplicity principles. Using the tools of operator theory and Fourier analysis, it is shown that the solution of the classical Tikhonov regularization problem can be derived from the regularized functional defined by a linear differential (integral) operator in the spatial (Fourier) domain. State-of-the-art research relevant to the regularization theory is reviewed, covering Occam's razor, minimum length description, Bayesian theory, pruning algorithms, informational (entropy) theory, statistical learning theory, and equivalent regularization. The universal principle of regularization in terms of Kolmogorov complexity is discussed. Finally, some prospective studies on regularization theory and beyond are suggested.