Accepted at CoRL 2026 Safety-Critical Control

Gradient-Free Neural Hamilton-Jacobi Reachability for Scalable Safety-Critical Control

Proceedings of the 10th Conference on Robot Learning (CoRL 2026)

Zeyuan Feng^*Stanford University
Ali Fuat Sahin^*EPFL
Santiago ThorupStanford University
Somil BansalStanford University

^*Equal contribution

Scroll

// 01 — Abstract

Learning Safe Reachability
Without Gradients

Hamilton-Jacobi (HJ) reachability provides a principled framework for synthesizing safety certificates and robust controllers for safety-critical robotic systems. However, applying reachability analysis to high-dimensional nonlinear systems remains challenging: classical grid-based solvers suffer from the curse of dimensionality, continuous-time neural solvers require accurate spatial value gradients, and reinforcement-learning-based approaches often suffer from weak boundary anchoring and non-stationary adversarial policy optimization.

We propose a discrete-time neural reachability framework for control-disturbance-affine systems that learns backward reachable tubes (BRTs) and backward reach-avoid tubes (BRATs) through Bellman-Isaacs value propagation. Our key idea is to combine equation-driven self-supervision with structured policy learning: rather than computing explicit PDE-gradients, we exploit the bang-bang structure of optimal safety interventions to construct approximate teacher actions from gradient-free value probes, converting adversarial actor learning into supervised policy learning.

To stabilize long-horizon value propagation, we leverage the learned actor to train the value function backward from the terminal boundary using a windowed temporal curriculum, where each window becomes the boundary condition for the next. Across benchmarks up to 80 dimensions, our method learns accurate value functions while improving stability over existing learning-based solvers. We further demonstrate observation-space scalability on F1-tenth racing with 16,000+ dimensional egocentric inputs, generalizing zero-shot to unseen tracks and transferring to a physical RC car.

No PDE-Gradients

Bang-bang probes replace ∇ₓV. No SIREN/PINN optimization noise, no gradient instability in high dimensions.

16K-Dim Vision Input

128×128 BEV occupancy grids fed directly — no handcrafted feature engineering required.

Stable to 80D

Windowed backward curriculum prevents bootstrap drift that collapses RL baselines at high dimensions.

#HJ Reachability #Robot Safety #Bellman-Isaacs #Bang-Bang #Sim-to-Real #CBF Filter

// 02 — Method

Discrete-Time Windowed Curriculum

Step 01

🪟

Temporal Partitioning

Horizon K split into windows. Each trains a specialized network, frozen as boundary condition for the next.

Step 02

⚡

Bang-Bang Probing

Optimal actions via gradient-free discrete value comparisons. Only 2(n_u+n_d) forward evaluations per state.

Step 03

🎓

Mixed TD Training

50/50 teacher (Bellman anchor) + student rollout targets. Oversamples near the terminal boundary.

Step 04

🔧

Rollout Finetuning

Asymmetric false-positive penalty + rollout anchors eliminate drift before freezing each window.

EQ.5 1 — Temporal Partitioning ▾

A single monolithic network causes representational overload for long-horizon problems with non-smooth reachable-set boundaries. We partition horizon K into windows — each with independent weights. Later-time frozen windows become boundary conditions for earlier windows.

V_θ(x,k) = b(x) if k=K | V^w_θ(x,k) if K−wW ≤ k < K−(w−1)W

↳Avoids monolithic representational overload
↳Frozen windows serve as stable BCs for earlier segments
↳Each network focuses capacity on a localized temporal segment

EQ.7 2 — Bang-Bang Probing ▾

Unlike PINNs, we infer optimal safety and disturbance actions via discrete value comparisons of forward-simulated next states — no analytic ∇ₓV needed. Only 2(n_u+n_d) evaluations per state convert unstable minimax RL into supervised regression.

û*_j = ū_j if C_{u,j} > 0 | u̲_j if C_{u,j} ≤ 0 C_{u,j} ≈ ΔV^u_j / Δt

↳No analytic gradient computation required
↳Converts minimax RL into stable supervised regression
↳Confidence weighting downweights flat-value regions

§3.2 3 — Mixed TD Targets ▾

50/50 blend of one-step teacher targets (Bellman-Isaacs anchor) and multi-step student rollout targets prevents compounding suboptimality. Time indices near the current boundary are oversampled to lock boundary-condition compliance.

y_mixed = 0.5 · y_teacher + 0.5 · y_student (M-step rollout)

↳Teacher targets preserve HJI recursion consistency
↳Student targets reduce policy drift over long rollouts

EQ.8 4 — Rollout Finetuning ▾

Before freezing each window, a finetuning phase calibrates the value network against empirical rollout targets under the current student policies. An asymmetric false-positive penalty suppresses states falsely predicted safe.

L_FT = L^student_TD + λ · Σ w_m(V_θ−G_m)² w_m=1+λ_fp·1{V>0, G<0}

↳Eliminates bootstrap drift across frozen window seams
↳Asymmetric penalty suppresses false-safe predictions

Figure 1 — Backward temporal curriculum with actor-critic learning

Method Overview: backward temporal curriculum with actor-critic learning

Left: Curriculum propagates backward from boundary condition (BC) at k=K. Each window trains, finetunes, then freezes as BC for the preceding window. Right: Within each step, the policy imitates bang-bang teacher probes and the value is updated with mixed TD(0)/TD(λ) targets.

// 03 — Results

Benchmark Comparisons

100K samples · 5 random seeds · FPR = dangerous over-optimism ↓ · FNR = over-conservatism ↓ · Succ ↑

Baselines: DeepReachMPC (continuous-time PINN) · RARL & ISAACS (actor–critic RL), all run from official codebases with tuned parameters.

10D Two-Vehicle Narrow Passage · Reach-Avoid · T = 8.0 s

Success Rate ↑

Ours 🏆99.65%

RL Baseline (ISAACS)3.70%

DeepReachMPC (PINN)2.50%

Method	FPR% ↓	FNR% ↓	Succ% ↑	Time(h) ↓
Ours	100† ± 0.00	0.00 ± 0.00	99.65 ± 0.31	4.0
DeepReachMPC	0.00 ± 0.00	100.0 ± 0.00	2.50 ± 0.90	5.0
RL Baseline	0.00 ± 0.00	93.40 ± 0.70	3.70 ± 0.72	5.5

† All evaluated initial states are inherently safe; 100% FPR reflects long-horizon (T=8s) policy execution errors, not value function over-optimism. The value correctly identifies states as safe — see §4.

13D Quadrotor Gate Avoidance · Avoid · T = 2.0 s · thrust-to-mass 20

Method	FPR% ↓	FNR% ↓	Succ% ↑	Time(h) ↓
Ours	0.50 ± 0.00	9.70 ± 1.00	0.50 ± 0.01	5.5
DeepReachMPC	2.00 ± 0.80	78.40 ± 14.00	0.06 ± 0.00	4.0
RL Baseline	30.38 ± 3.39	0.00 ± 0.00	0.00 ± 0.00	5.5

The 0.50% success rate reflects physical reality — most random initial states are dynamically doomed by gravity (z-velocity constraint). Our policy outperforms DeepReachMPC's built-in MPC solver (0.03%) and a carefully tuned MPPI controller (0.21%). RL Baseline FPR of 30.38% signals dangerous over-optimism.

40D / 80D Publisher-Subscriber Network · Reach · ground-truth MSE available

System	Method	FPR% ↓	FNR% ↓	MSE ↓
40D Publisher-Subscriber Network
40D	Ours	0.000 ± 0.000	9.134 ± 0.693	0.009 ± 0.002
40D	DeepReachMPC	2.648 ± 0.310	0.001 ± 0.001	0.007 ± 0.001
40D	RL Baseline	23.306 ± 0.397	76.228 ± 0.602	1.717 ± 0.086
80D Publisher-Subscriber Network
80D	Ours	0.219 ± 0.429	14.263 ± 1.884	0.072 ± 0.016
80D	DeepReachMPC	2.115 ± 0.145	0.000 ± 0.000	0.026 ± 0.001
80D	RL Baseline	25.581 ± 0.941	78.696 ± 0.731	5.886 ± 0.248

Our generalized two-player zero-sum solver matches DeepReachMPC's MSE — despite DeepReachMPC being engineered for single-player optimal control. The RL baseline collapses entirely at 80D (MSE > 5.8, FNR > 78%).

Component-Wise Ablation · Narrow Passage Success Rate ↑

Full method 🏆99.65%

− Asymmetric FP weighting85.53%

− Window fine-tuning65.57%

− Windowing (monolithic net)9.32%

Both pillars are load-bearing

Replacing the windowed architecture with a single monolithic network collapses success to 9.32% — a 90-point drop. Omitting the per-window fine-tuning phase costs 34 points. Temporal partitioning and boundary-correction fine-tuning are therefore the two components most critical to long-horizon reachable-set accuracy.

Policy Optimization Objective · Narrow Passage

Policy Objective	FPR% ↓	FNR% ↓	Succ% ↑	Time (min) ↓
Policy Gradient — 1 Step	99.58 ± 0.09	0.79 ± 0.74	92.20 ± 6.11	265.5
Policy Gradient — 3 Step	93.57 ± 8.71	0.25 ± 0.13	90.15 ± 6.87	307.4
Policy Gradient — 5 Step	99.75 ± 0.09	0.28 ± 0.83	80.81 ± 3.85	400.1
Teacher-Student (Ours)	100† ± 0.0	0.00 ± 0.00	99.65 ± 0.31	262.4

The teacher–student objective converges faster and far more consistently across seeds than policy-gradient alternatives, and removes the need to tune the maximum rollout horizon m — a hyperparameter that dominates PG training stability. † See the Narrow Passage tab for the FPR footnote.

Teacher Mixing Fraction α · Dubins3D · 20,000 test states

Teacher Mixing α	ΔSucc% ↓	ΔCost Mean ↓	ΔCost Std ↓
Pure Student (α=0.0)	0.2210 ± 0.0859	0.0356 ± 0.0015	0.0772 ± 0.0021
Pure Teacher (α=1.0)	0.3362 ± 0.0608	0.0274 ± 0.0013	0.0643 ± 0.0025
Decaying (α: 1→0)	0.2267 ± 0.0607	0.0299 ± 0.0070	0.0699 ± 0.0061
Constant α=0.5 (Ours)	0.2056 ± 0.0856	0.0312 ± 0.0018	0.0706 ± 0.0026

Metrics report the absolute difference vs. the numerical solution. ΔSucc = success-rate gap; ΔCost Mean / ΔCost Std = differences in mean and std of rollout cost. Constant α=0.5 achieves the closest match to the numerical baseline in success rate. Teacher-only yields the best rollout cost but the largest success-rate gap, suggesting over-reliance on teacher targets can bias policy learning.

Method	False Positives ↓	False Negatives ↓
Ours	206	35
OptimizedDP	1,569	10

Pursuer Policy	Evader: MPPI	Evader: Ours
Ours	88%	42%
MPPI	36%	0%

Time Discretization Sensitivity · Narrow Passage · 20,000 test states

Step Size Δt	FPR% ↓	FNR% ↓	Succ% ↑	Time (h) ↓
Δt = 0.025s	100.0	0.0	82.01	11
Δt = 0.050s	100.0	0.0	94.29	5
Δt = 0.075s	100.0	0.0	93.49	4
Δt = 0.100s	100.0	0.0	96.31	3
Δt = 0.125s ★	100.0	0.0	99.99	2.5
Δt = 0.150s	100.0	0.0	99.98	1.5
Δt = 0.300s	0.0	100.0	78.75	1

Performance is robust across a wide range of Δt. Moderate steps (0.1–0.15s) achieve the best success rates with substantially reduced training time. Very small Δt (0.025s) expands the curriculum horizon, increasing bootstrap accumulation and training cost. Excessively coarse steps (0.3s) flip FNR to 100%, indicating the local dynamics approximation breaks down. ★ = used in main experiments.

// 04 — Video Demonstrations

Real-Time Testing

Side-by-side: HJ safety filter vs unfiltered nominal controller.

F1Tenth with Safety Filter

10 laps · zero collisions · avg 1.07 m/s on unseen physical track. All compute runs offboard over WiFi with ~50 ms round-trip.

✓ Filtered — 0 collisions

F1Tenth without Safety Filter

Unfiltered MPPI tracks racing line at max speed — 3 collisions within the very first lap on the physical track.

✗ Unfiltered — 3 crashes lap 1

// 05 — F1Tenth Sim-to-Real

Vision-Based Racing & Physical Deployment

BRT safety filter for high-speed autonomous racing (max 10 m/s). The model ingests a 128×128 egocentric BEV occupancy image + 5D proprioceptive state + time (16,390D total) plus a disturbance scale s_d ∈ [0, 0.3]. BEV resolution scales with vehicle speed to guarantee sufficient look-ahead. Trained jointly across 20 tracks and evaluated on 3 held-out tracks (~120 m × 120 m each). At test time the learned value function acts as a Discrete-Time Control Barrier Function (DCBF) filter over a deliberately poorly-tuned nominal MPPI controller.

0 / 50

Collisions with safety filter (simulation)

50 evaluation trials of 40 s each. Avg speed 9.25 m/s (in-distribution) and 9.17 m/s (OOD tracks), zero-shot to 3 held-out tracks under disturbance scale s_d = 0.1.

56% / 48%

Collision rate without filter (simulation)

Unfiltered MPPI on in-distribution and OOD tracks respectively. Failures at high slip angles near sharp turns. LiDAR insufficient — BEV is essential.

10 laps

Zero collisions — physical RC car (Rosmaster R2)

Avg 1.07 m/s at hardware limit (1.5 m/s), after finetuning on 20 scaled-down tracks. s_d = 0.2 absorbs ~50 ms WiFi latency + sim-to-real gap on a previously unseen track.

3 crashes

Unfiltered policy — first lap only

Nominal MPPI blindly tracks racing line at max speed, colliding within its very first lap on the physical track.

Hardware Setup

🚗

Platform

Rosmaster R2 RC scale car

📡

Localization

Vicon motion capture system

🔦

Onboard Sensor

7 Hz LiDAR (map construction)

💻

Offboard Compute

GTX 1060 · 12-core 3.3 GHz

📶

Communication

WiFi · ~50 ms round-trip

👁️

Input Dimensionality

128×128 BEV + 5D proprioceptive + time = 16,390D

// 06 — Citation

BibTeX

BibTeX Reference

@inproceedings{feng2026gradfreereachability,
  title     = {Gradient-Free Neural Hamilton-Jacobi Reachability
               for Scalable Safety-Critical Control},
  author    = {Feng, Zeyuan and Sahin, Ali Fuat and
               Thorup, Santiago and Bansal, Somil},
  booktitle = {Proceedings of the 10th Conference on Robot Learning (CoRL)},
  year      = {2026},
  note      = {* Equal contribution}
}