Regensburg 2019 – wissenschaftliches Programm
SOE 12.3: Poster
Dienstag, 2. April 2019, 16:00–19:00, Poster A
Deep Reinforcement Learning in World-Earth-System Models Exploring Sustainable Behavior — •Felix Strnad1,2, Wolfram Barfuß1, Jonathan F. Donges1,3, and Jobst Heitzig1 — 1Potsdam Institute for Climate Impact Research, Germany — 2Department of Physics, University of Göttingen, Germany — 3Stockholm Resilience Centre, Stockholm University, Sweden
Pathways to global sustainability need to account for critical feedbacks between the socio-cultural World and the biophysical Earth system. These feedbacks may require novel, yet undiscovered global policies for the governance leading towards a safe and just operating space. Currently the combination of agent-based modeling, reinforcement learning and deep neural networks, called Deep Reinforcement Learning (DRL), has become increasingly popular. DRL-algorithms have been shown to learn policies up to super-human performance in a variety of different environments.
In this work, we apply DRL within stylized World-Earth models. We developed an agent that is able to act and learn in variable manageable non-linear complex environments. We trained our agent with a deep Q-network (DQN) using experience replay and periodically updated target networks. We systematically investigated the effect of various parameters for the learning success, such as the discount factor, the training data set size or the exploration-exploitation trade-off. By using this optimized parameter set, we find that our agent is able to learn novel policies towards sustainable regions in multiple conceptual models of the World-Earth system.