Interpolation Models with Multiple Hyperparameters

David J C MacKay and Ryo Takeuchi

A traditional interpolation model is characterized by the choice of regularizer applied to the interpolant, and the choice of noise model. Typically, the regularizer has a single regularization constant \alpha, and the noise model has a single parameter beta. The ratio \alpha/beta alone is responsible for determining globally all these attributes of the interpolant: its `complexity', `flexibility', `smoothness', `characteristic scale length', and `characteristic amplitude'. We suggest that interpolation models should be able to capture more than just one flavour of simplicity and complexity. We describe Bayesian models in which the interpolant has a smoothness that varies spatially. We emphasize the importance, in practical implementation, of the concept of `conditional convexity' when designing models with many hyperparameters. We apply the new models to the interpolation of neuronal spike data and demonstrate a substantial improvement in generalization error.

postscript (Cambridge UK).

postscript (Canada mirror).


David MacKay's: home page, publications. bibtex file.
Canadian mirrors: home page, publications. bibtex file.