Berlin 2024 – wissenschaftliches Programm

Bereiche | Tage | Auswahl | Suche | Aktualisierungen | Downloads | Hilfe

SOE: Fachverband Physik sozio-ökonomischer Systeme

SOE 2: Focus Session: Machine Learning for Complex Socio-economic Systems

SOE 2.5: Vortrag

Montag, 18. März 2024, 11:00–11:15, MA 001

Towards a complex systems theory of attention? — •Claudius Gros — Institute for Theoretical Physics, Goethe University Frankfurt

The attention mechanism is at the core of the current AI hype. It powers transformers and hence all modern large language models, such as GPT or LLaMA. Classical deep learning model are optimized for information processing, whereas attention allows for information routing. It is argued, that the modeling techniques used complex systems theory and physics can contribute to an understanding of what is going on inside transformers. On this background an introduction to attention is presented.