Our paper on risk-adaptive reinforcement learning with differentiable CVaR barrier functions has been accepted to IEEE Robotics and Automation Letters (RA-L)! We integrate a differentiable safety filter into reinforcement learning to jointly learn control policies, risk allocation, and safety margins for navigation under uncertainty. The approach provides probabilistic safety guarantees while reducing unnecessary conservatism, and demonstrates robust performance across dynamic environments, different robot models, and out-of-distribution scenarios.
DiffCVaR: Reinforcement Learning for Risk Adaptation via Differentiable CVaR Barrier Functions
Authors: Xinyi Wang, Taekyung Kim, Bardh Hoxha, Georgios Fainekos, Dimitra Panagou
Abstract: Planning through crowded environments under uncertain obstacle motions remains difficult, as stochastic interactions often induce overly conservative behavior or reduced efficiency. To address this challenge, we propose an end-to-end risk adaptation framework for crowd navigation under obstacle-motion uncertainty modeled by a Gaussian mixture model. The framework combines reinforcement learning~(RL) with a differentiable quadratic-program safety layer based on Conditional Value-at-Risk~(CVaR) barrier functions, jointly learning nominal control input, risk level, and safety margin and enforcing explicit probabilistic safety constraints. This design enables context-aware adaptation, promoting efficient behavior while invoking caution only when necessary. We conduct extensive evaluations in dynamic, uncertain, and crowded environments across varying obstacle densities and robot models, and further assess generalization under three out-of-distribution cases. Comparisons across optimization-based, RL-based, and integrated RL and optimization methods are provided, and the proposed method is shown to deliver the strongest overall performance in safety, efficiency, and generalization under uncertainty.
