#
Compactness of variational approximations

##
David J C MacKay, Richard Turner, and
Maneesh Sahani

Let's approximate a complicated distribution P(x) by a simpler
distribution Q(x), possibly a separable distribution.
It's often the case that variational free energy minimization
(also known as mean field) leads to an approximating distribution
Q that is `more compact' than the true distribution.

Is there in fact a **theorem** that we could prove along the lines of
`optimized $Q$ is **always** more compact'?

We show, with a counterexample,
that the folk theorem about variational
approximations being `more compact' is not *always* true.

postscript (Cambridge UK).

postscript (Canada mirror).

pdf (Cambridge UK).

pdf (Canada mirror).

All postscript files are compressed with gzip -
see this page for advice about gzip, if needed.

related publications.

David MacKay's:
home page,
publications.
bibtex file.

Canadian mirrors:
home page,
publications.
bibtex file.