Regensburg 2019 – scientific programme

Parts | Days | Selection | Search | Updates | Downloads | Help

SOE: Fachverband Physik sozio-ökonomischer Systeme

SOE 12: Poster

SOE 12.3: Poster

Tuesday, April 2, 2019, 16:00–19:00, Poster A

Deep Reinforcement Learning in World-Earth-System Models Exploring Sustainable Behavior — •Felix Strnad^1,2, Wolfram Barfuß¹, Jonathan F. Donges^1,3, and Jobst Heitzig¹ — ¹Potsdam Institute for Climate Impact Research, Germany — ²Department of Physics, University of Göttingen, Germany — ³Stockholm Resilience Centre, Stockholm University, Sweden

Pathways to global sustainability need to account for critical feedbacks between the socio-cultural World and the biophysical Earth system. These feedbacks may require novel, yet undiscovered global policies for the governance leading towards a safe and just operating space. Currently the combination of agent-based modeling, reinforcement learning and deep neural networks, called Deep Reinforcement Learning (DRL), has become increasingly popular. DRL-algorithms have been shown to learn policies up to super-human performance in a variety of different environments.

In this work, we apply DRL within stylized World-Earth models. We developed an agent that is able to act and learn in variable manageable non-linear complex environments. We trained our agent with a deep Q-network (DQN) using experience replay and periodically updated target networks. We systematically investigated the effect of various parameters for the learning success, such as the discount factor, the training data set size or the exploration-exploitation trade-off. By using this optimized parameter set, we find that our agent is able to learn novel policies towards sustainable regions in multiple conceptual models of the World-Earth system.