T4 – Conditioning reservoir operations on the state of the climate system

Short description

Water reservoir operations are increasingly challenged by both short-term hydroclimatic variability and long-term oscillations and climate change. Despite this, most existing reservoir operating rules are still based on a limited set of variables, typically including the day of the year, reservoir storage, and, in some cases, the previous day inflow [1]. These rule curves are generally designed to perform satisfactorily under average conditions, but they may become ineffective under extreme events or in the presence of nonstationary hydroclimatic trends.

Recent advances in weather and climate data availability offer new opportunities to improve reservoir operations. In particular, spatially distributed hydroclimatic information can provide a more complete representation of both the local hydrological state and the broader regional or global climate conditions influencing inflows and water availability [2]. Leveraging this richer information set may enhance the adaptability and robustness of operational decisions, especially during floods, droughts, and other extreme hydroclimatic events. However, effectively incorporating large-scale spatial weather and climate information into reservoir operation remains a major challenge, as this requires handling high-dimensional state spaces.

This thesis aims to develop a deep reinforcement learning approach to advance water reservoir operations by exploiting spatially distributed weather and climate data. Specifically, the work will focus on approximating the action-value function through convolutional neural networks applied to the hydroclimatic state space. The final objective is to assess whether integrating hydroclimatic predictors into DRL-based reservoir control can improve operational performance and resilience under extreme events.

PROPOSED ACTIVITIES

Literature Review

  • Conduct a systematic literature review on adaptive reservoir and water systems operation under hydroclimatic variability and climate change, with a focus on data-driven and learning-based approaches
  • Analyze recent studies on the application of artificial intelligence to environmental and water-related control systems, in order to identify suitable methodologies, understand current research gaps, and assess the broader implications of the proposed work
  • Review the methodological framework to be used, including reinforcement learning and deep reinforcement learning, convolutional neural networks for extracting information from spatial weather and climate data

Data Processing and Analysis

  • Collect, preprocess, and integrate spatially distributed weather, climate, and system-state data to construct the input space for the decision-making model
  • Explore the spatial and temporal structure of the input data, including variability, extremes, and possible nonstationary trends associated with climate change, to ultimately define performance metrics measuring adaptability to climate change and extreme events
  • Evaluate different strategies for feature extraction and dimensionality reduction, with particular attention to convolutional neural networks for processing gridded weather and climate information

RL-Based Control and Performance Assessment

  • Develop and train a reinforcement learning framework [3] for adaptive reservoir operation under different hydroclimatic conditions with state representation derived from the data analysis
  • Design and compare alternative learning architectures for integrating spatial weather and climate information into the control policy
  • Evaluate (based on previously defined metrics and indicators) the learned policies against baseline operating rules and simplified information settings

References

  1. Hejazi, M.I., Cai, X., Ruddell, B.L., 2008. The role of hydrologic information in reservoir operation learning from historical releases. Adv. Water Resour. 31 (12), 1636–1650. http://dx.doi.org/10.1016/j.advwatres.2008.07.013.
  2. Giuliani, M., M. Zaniolo, A. Castelletti, G. Davoli, and P. Block (2019), Detecting the state of the climate system via artificial intelligence to improve seasonal forecasts and inform reservoir operations, Water Resources Research, 55
  3. Castelletti, Andrea, Francesca Pianosi, and Marcello Restelli. “A multiobjective reinforcement learning approach to water resources systems operation: Pareto frontier approximation in a single run.” Water Resources Research 49.6 (2013): 3476-3486.

Relevant courses and knowledge

Natural Resources Management 1+2 (optional: machine learning or other AI courses from computer science)

Number of students

1

Requisites

The student should be comfortable with data handling. Proficient coding skills (Python preferred) and interest in the algorithmical aspects are needed.