Bereiche | Tage | Auswahl | Suche | Aktualisierungen | Downloads | Hilfe
AKPIK: Arbeitskreis Physik, moderne Informationstechnologie und Künstliche Intelligenz
AKPIK 2: Machine Learning Prediction and Optimization Tasks
AKPIK 2.5: Vortrag
Dienstag, 10. März 2026, 10:30–10:45, BEY/0127
Phase Transitions reveal Accuracy Hierarchies in Deep Learning — •Ibrahim Talha Ersoy1, Andrés Fernando Cardozo Licha2, and Karoline Wiesner1 — 1Universität Potsdam, Institut für Astronomie und Physik, Potsdam, Deutschland — 2Universidade Federal Fluminense, Instituto de Física, Niterói, Brazil
Training Deep Neural Networks relies on the model converging on a high-dimensional, non-convex loss landscape toward a good minimum. However, much of the phenomenology of training remains ill understood. We focus on three seemingly disparate phenomena: the observation of phase transitions akin to statistical physics, the ubiquity of saddle points, and mode connectivity which is key for the active research area of model merging. We bring these into a single explanatory framework, that of the geometry of the loss and error landscapes. We show analytically that phase transitions in DNN learning are governed by saddle points in the loss landscape. Furthermore, we present a simple, easy to implement and fast algorithm, using the L2 regularizer as a tool, to explore the geometry of error landscapes. We demonstrate its use for efficiently finding paths connecting global minima by confirming the mode connectivity for DNNs trained on the MNIST data set to then use it to show numerically that saddle points in DNN loss landscapes mark transitions between distinct models that encode distinct digits of the MNIST data. Our work establishes the geometric origin of key DNN training phenomena and reveals hierarchically ordered accuracy basins analogous to phases in statistical physics.
Keywords: Neural Networks; Phase Transitions; Learning Hierarchies; Information Geometry