Bereiche | Tage | Auswahl | Suche | Aktualisierungen | Downloads | Hilfe
SOE: Fachverband Physik sozio-ökonomischer Systeme
SOE 17: Focus Session: Physics of AI II (joint session SOE/DY)
SOE 17.9: Vortrag
Freitag, 13. März 2026, 12:15–12:30, GÖR/0226
Statistical physics of deep learning: Optimal learning of a multi-layer perceptron near interpolation — Jean Barbier1, Francesco Camilli1, Minh-Toan Nguyen1, Mauro Pastore1, and •Rudy Skerk2 — 1The Abdus Salam International Centre for Theoretical Physics, Strada Costiera 11, 34151 Trieste, Italy — 2International School for Advanced Studies, Via Bonomea 265, 34136 Trieste, Italy
We address a long-standing question in statistical physics by analysing the supervised learning of a multi-layer perceptron, beyond narrow models and kernel methods. Crucially, (i) the width scales with input dimension, making the model more prone to feature learning than ultra-wide networks and more expressive than narrow ones; and (ii) we work in the interpolation regime where trainable parameters and data are comparable, forcing task-specific adaptation. In a matched teacher-student setting we establish the fundamental limits for learning random deep-network targets and identify the sufficient statistics that an optimally trained network acquires as data increases. A rich phenomenology appears with multiple learning transitions: with enough data optimal performance arises via model "specialisation", yet practical algorithms can be trapped in theory-predicted suboptimal solutions. Specialisation occurs inhomogeneously across layers, propagating from shallow towards deep ones, but also across neurons in each layer. The Bayesian-optimal analysis thus clarifies how depth, nonlinearity and finite (proportional) width shape feature learning, with implications beyond this idealised setting.
Keywords: Spin-glass theory; Multi-layer percepron; Supervised learning; Statistical-to-computational gap; HCIZ integrals