Bereiche | Tage | Auswahl | Suche | Aktualisierungen | Downloads | Hilfe

SOE: Fachverband Physik sozio-ökonomischer Systeme

SOE 17: Focus Session: Physics of AI II (joint session SOE/DY)

SOE 17.3: Vortrag

Freitag, 13. März 2026, 10:15–10:30, GÖR/0226

Online Learning Dynamics and Neural Scaling Laws for a Perceptron Classification Problem — •Yoon Thelge, Marcel Kuhn, and Bernd Rosenow — Institute for Theoretical Physics, University of Leipzig, 04103 Leipzig, Germany

Understanding neural scaling laws and emergence of power law generalisations remains a central challenge in learning dynamics. A natural setting for analysing this behaviour is the online-learning dynamics of a perceptron trained in a teacher*student scenario, where in the thermodynamic limit, the generalisation error exhibits characteristic power-law decay. In realistic classification problems, the teacher is a discrete classifier, while standard gradient-based training requires the student to have continuous outputs. Thus, in practically relevant settings the student is necessarily mismatched to the discrete teacher, a regime that is less well understood. We study this regime for a perceptron with a sign-activation teacher and an error-function student. We derive coupled differential equations for the evolution of the relevant order parameters and verify them via numerical integration and SGD simulations. For fixed learning rates, the generalisation error converges to zero as a power-law with respect to the number of training examples with an exponent of -1/3. The onset of this asymptotic regime shifts with the learning rate, and the generalisation at the onset scales with exponent -1/2, motivating the use of learning-rate schedules to enhance the effective asymptotic decay.

Keywords: Neural Networks; Online learning; Neural scaling laws; Statistical Mechanics