Regensburg 2004 – wissenschaftliches Programm
Bereiche | Tage | Auswahl | Suche | Downloads | Hilfe
DY: Dynamik und Statistische Physik
DY 21: Neural Networks
DY 21.4: Vortrag
Dienstag, 9. März 2004, 11:00–11:15, H3
A statistical mechanics approach to approximate analytical Bootstrap averages — •Dörthe Malzahn1 and Manfred Opper2 — 1Institut für Mathematische Stochastik, Universität Karlsruhe, 76128 Karlsruhe — 2Neural Computing Research Group, Aston University, Birmingham B4 7ET, United Kingdom
Information processing systems are often described by models with a large number of degrees of freedom which interact by a random energy function. We consider the problem of learning from example data where the randomness is induced by the data. Bootstrap is a general method to evaluate the average learning performance. It estimates averages over the true data generating distribution (the disorder) by an average over an ensemble of surrogate data sets which are generated by sampling from a set of available data. This offers the advantage that the data generating process is known and can be controlled by the experimenter. Using tools from the physics of disordered materials, we develop a general framework to calculate approximate analytical Bootstrap averages. We apply our method to the Bootstrap of Gaussian process models which can be understood as (Bayesian) feed-forward neural networks with infinitely many neurons in the hidden layer. Our method for the analytical calculation of Bootstrap averages works on real data and yields quantitative results which are reliable and faster to compute than Monte-Carlo averages. The results can be used to evaluate and optimize the learning performance.