HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS		QUICK SEARCH:   [advanced] Author: Keyword(s): Year:  Vol:  Page:

This Article Full Text Full Text (PDF) Alert me when this article is cited Alert me if a correction is posted Services Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Vehtari, A. Articles by Lampinen, J. Search for Related Content PubMed PubMed Citation Articles by Vehtari, A. Articles by Lampinen, J.

Bayesian Model Assessment and Comparison Using Cross-Validation Predictive Densities

Aki.Vehtari{at}hut.fi, Laboratory of Computational Engineering, Helsinki University of Technology, FIN-02015, HUT, Finland

Jouko Lampinen

Jouko.Lampinen{at}hut.fi, Laboratory of Computational Engineering, Helsinki University of Technology, FIN-02015, HUT, Finland

In this work, we discuss practical methods for the assessment, comparison, and selection of complex hierarchical Bayesian models. A natural way to assess the goodness of the model is to estimate its future predictive capability by estimating expected utilities. Instead of just making a point estimate, it is important to obtain the distribution of the expected utility estimate because it describes the uncertainty in the estimate. The distributions of the expected utility estimates can also be used to compare models, for example, by computing the probability of one model having a better expected utility than some other model. We propose an approach using cross-validation predictive densities to obtain expected utility estimates and Bayesian bootstrap to obtain samples from their distributions. We also discuss the probabilistic assumptions made and properties of two practical cross-validation methods, importance sampling and k-fold cross-validation. As illustrative examples, we use multilayer perceptron neural networks and gaussian processes with Markov chain Monte Carlo sampling in one toy problem and two challenging real-world problems.

This article has been cited by other articles:

				M. Grzegorczyk, D. Husmeier, K. D. Edwards, P. Ghazal, and A. J. Millar Modelling non-stationary gene regulatory processes with a non-homogeneous Bayesian network and the allocation sampler Bioinformatics, September 15, 2008; 24(18): 2071 - 2078. [Abstract] [Full Text] [PDF]

				M. Plummer Penalized loss functions for Bayesian model comparison Biostat., July 1, 2008; 9(3): 523 - 539. [Abstract] [Full Text] [PDF]

				T. Miazhynskaia, S. Fruhwirth-Schnatter, and G. Dorffner Neural Network Models for Conditional Distribution Under Bayesian Analysis Neural Comput., February 1, 2008; 20(2): 504 - 522. [Abstract] [Full Text] [PDF]