Dresden 2026 – wissenschaftliches Programm

Bereiche | Tage | Auswahl | Suche | Aktualisierungen | Downloads | Hilfe

DY: Fachverband Dynamik und Statistische Physik

DY 14: Machine Learning in Dynamics and Statistical Physics II

DY 14.5: Vortrag

Montag, 9. März 2026, 16:00–16:15, HÜL/S186

Microscopy on Autopilot: Self-Supervised Transformers for Feature Detection and Control — •Damián Baláž, Gianmarco Ducci, Christoph Scheurer, Karsten Reuter, and Hendrik H. Heenen — Fritz-Haber-Institut der MPG, Berlin

The evaluation of microscopy experiments often relies on manual inspection or supervised machine learning. The former is inefficient, whereas the latter requires extensive labeling and may introduce human bias. Self-supervised learning, by contrast, learns from raw image data, capturing intrinsic visual patterns without the need for manual annotation. This improves generalization and objectivity, making it ideal for complex and dynamic microscopy data. Motivated by these advantages, we use a pre-trained self-supervised machine learning model (DINO), based on vision transformer architecture. This constitutes our central tool for feature detection and temporal analysis in microscopy experiments.

We demonstrate the versatility of our approach for two microscopy experiments: i) observing graphene flake growth on liquid copper ii) tracking crack formation in a cobalt oxide catalyst. In both cases, the model enables label-free, qualitative monitoring by identifying related structures based on similarity in the learned feature space. Beyond using it for analysis, we show how the same feature space representations can be used to predict experimental dynamics to autonomously steer processes toward desired targets via planning in feature space. Our findings highlight the potential of self-supervised vision models for real-time analysis and control in microscopy-based experiments.

Keywords: Machine learning; Self-supervised learning; Autonomous experiments; Computer Vision; Graphene growth