1. Home
  2. RIP
  3. View Record
https://www.nationalacademies.org/trb/events

A Reinforcement Learning Framework for Dynamic Inland Waterway Maintenance Under Stochastic Shoaling and Annual Budget Allocation

This research proposes a dynamic, data-driven framework for long-term inland waterway maintenance planning that integrates reinforcement learning (RL), and stochastic modeling. Unlike traditional models that assume deterministic sedimentation, known multi-year budgets, and static decision horizons, the research team models shoaling as a stochastic process, budgets as annually realized random variables, and infrastructure deterioration as a gradual, condition-dependent process. The core of the methodology is an infinite-horizon sequential decision model that makes year-by-year dredging and lock maintenance decisions using RL. Dredging is modeled as a continuous decision variable, and policy learning is guided by a custom-designed simulation environment that reflects realistic physical and institutional constraints. The team trains RL agents using Proximal Policy Optimization (PPO). This work addresses the curse of dimensionality that limits conventional optimization techniques by learning generalizable policies rather than enumerating all possible scenarios. By finding the solution across various uncertainty regimes, the team provides both methodological insights and practical guidance for agencies such as the U.S. Army Corps of Engineers. The resulting framework offers a robust and adaptive tool for managing long-term infrastructure investment under uncertainty