FAITH: Feasibility-Aware Safety-Filtered RL for High-Dimensional Systems

2026-10-08

AI Summary

This study addresses the objective conflicts inherent in safe reinforcement learning and the model dependence and myopia of traditional safety filters by proposing a model-free safety filtering framework. The method approximates the safety value function using a feedforward network to perform filtering, enabling the task policy to optimize returns under dynamic intervention. An amortized design achieves feasibility awareness and minimal intervention, eliminating competing updates while minimizing peak harm when constraints are infeasible. Experiments demonstrate that this framework effectively balances high safety guarantees with task performance across double integrator, Safety Gym, and 29-degree-of-freedom humanoid robot simulations. Furthermore, the proposed approach is successfully deployed on a physical Unitree G1 humanoid robot, validating its practical applicability in real-world settings.

Abstract

Safe reinforcement learning commonly places safety and task performance in the same policy objective, where they can introduce competing updates. Safety filters separate them at action execution, but classical designs require an analytic safety function and dynamics model, and standard minimal-intervention filters are myopic to long-horizon task return because they minimize only instantaneous action deviation. Hard projections are also undefined when no safe action exists. We present FAITH, a feasibility-aware, model-free framework that approximates the optimal state-action safety value and amortizes minimal-intervention filtering with a feedforward network. The task policy optimizes the task return through the filtered dynamics, which recovers the feasible constrained problem without a competing safety term in the task-policy update. When no action satisfies the learned safety condition, the same filter approaches the action with minimum predicted peak harm. On a double integrator example and a Safety Gym environment, FAITH achieves the highest return among methods with no feasible-start violations and matches the lowest harm from infeasible starts. On a 29-DoF humanoid, it reaches a 99.95% safety rate while retaining 97% of the unfiltered return in Walking-Avoid, and obtains the highest measured safety rate in Push-Avoid by learning to sacrifice balancing and fall away from the protected region. The same policies are also demonstrated on a real-world Unitree G1 humanoid.

Problem

• Safe Reinforcement Learning
• Safety Filter
• High-Dimensional Systems
• Minimal Intervention
• Feasibility

Innovation

• Safe Reinforcement Learning
• Safety Filter
• Model-Free
• Feasibility-Aware
• Humanoid Robot