#
Bayesian Methods for Neural Networks: Theory and Applications

##
David J C MacKay

Neural networks are parameterized non-linear models used for empirical
regression and classification modelling. Their flexibility makes
them able to discover more general relationships in data than traditional
statistical models.
Bayesian probability theory provides a unifying framework for
data modeling which offers several benefits.
First, the {\bf overfitting
problem} can be solved by using Bayesian methods to control model
complexity. Bayesian model comparison can be used for example
to optimize weight decay rates, and to infer automatically which
are the relevant input variables for a problem.
%
Second, probabilistic modelling handles
uncertainty in a natural manner. There is a unique prescription, {\bf
marginalization}, for incorporating uncertainty about parameters into
predictions; this procedure yields better predictions. Third, we can
define more sophisticated probabilistic models which are able to
extract more information from data.

I will discuss the practical
application of these methods to a problem involving
the prediction of a series of building energy loads from a
series of environmental variables.

I will also discuss the application of Bayesian methods to neural network
classifiers.

Finally I will describe the implementation of a probabilistic
regression model in BUGS. BUGS is a program that carries out
Bayesian inference on statistical problems using a simulation
technique known as Gibbs sampling.

postscript.

How to cite this document.

It's not been published in a journal.
You may find that this published article is sufficiently similar that you
can cite it:
@article{MacKay95:network,
KEY ="MacKay",
AUTHOR ="D. J. C. MacKay",
TITLE ="Probable Networks and Plausible Predictions --- A
Review of Practical {B}ayesian Methods for Supervised
Neural Networks",
journal ="Network: Computation in Neural Systems",
volume = 6,
YEAR =1995,
pages = "469-505",
ANNOTE ="Date submitted: 1994; Date accepted: 1994; Collaborating
institutes: none"}

If not, cite the URL of this abstract page.

David MacKay's:
home page,
publications.
bibtex file.