Probable Networks and Plausible Predictions -
A Review of Practical
Bayesian Methods for Supervised Neural Networks
David J C MacKay
Bayesian probability theory provides a unifying framework for
data modelling. In this framework the overall aims are to
find models that are well-matched to the data, and to use
these models to make optimal predictions. Neural network
learning is interpreted as an inference of the most
probable parameters for the model, given the training data.
The search in model space (i.e., the space of architectures,
noise models, preprocessings, regularizers and weight decay
constants) can also then be treated as an inference problem,
in which we infer the relative probability of alternative
models, given the data. This review describes practical
techniques based on Gaussian approximations for implementation
of these powerful methods for controlling,
comparing and using adaptive networks.
(Final version, 1 Feb 1995). To appear in Network, IOPP.
41 pages. 508K.