Submitted to CoRL 2026 · Safety-Critical Control · Anonymous

Gradient-Free Neural Hamilton-Jacobi Reachability for Scalable Safety-Critical Control

10th Conference on Robot Learning (CoRL 2026)

Anonymous Author(s) · Double-Blind Review

Paper PDF Anonymous Code Live Sandbox BibTeX

Scroll

// 01 — Abstract

Learning Safe Reachability
Without Gradients

Hamilton-Jacobi (HJ) reachability provides a principled framework for synthesizing safety certificates and robust controllers for safety-critical robotic systems. However, applying reachability analysis to high-dimensional nonlinear systems remains challenging: classical grid-based solvers suffer from the curse of dimensionality, continuous-time neural solvers require accurate spatial value gradients, and RL-based approaches suffer from weak boundary anchoring.

We propose a discrete-time neural reachability framework for control-disturbance-affine systems that learns backward reachable tubes (BRTs) and BRATs through Bellman-Isaacs value propagation. Our key idea: exploit the bang-bang structure of optimal safety interventions to construct teacher actions from gradient-free value probes, converting adversarial actor learning into supervised policy learning.

To stabilize long-horizon value propagation we train the value function backward from the terminal boundary using a windowed temporal curriculum. Across benchmarks up to 80 dimensions our method improves stability over existing solvers. We further scale to F1-tenth racing with 16,000+ dimensional egocentric inputs, achieving zero-shot sim-to-real transfer.

No PDE-Gradients

Bang-bang probes replace ∇ₓV. No SIREN/PINN optimization noise, no gradient instability in high dimensions.

16K-Dim Vision Input

128×128 BEV occupancy grids fed directly — no handcrafted feature engineering required.

Stable to 80D

Windowed backward curriculum prevents bootstrap drift that collapses RL baselines at high dimensions.

#HJ Reachability #Robot Safety #Bellman-Isaacs #Bang-Bang #Sim-to-Real #CBF Filter

// 02 — Method

Discrete-Time Windowed Curriculum

Step 01

🪟

Temporal Partitioning

Horizon K split into windows. Each trains a specialized network, frozen as boundary condition for the next.

Step 02

⚡

Bang-Bang Probing

Optimal actions via gradient-free discrete value comparisons. Only 2(n_u+n_d) forward evaluations per state.

Step 03

🎓

Mixed TD Training

50/50 teacher (Bellman anchor) + student rollout targets. Oversamples near the terminal boundary.

Step 04

🔧

Rollout Finetuning

Asymmetric false-positive penalty + rollout anchors eliminate drift before freezing each window.

EQ.5 1 — Temporal Partitioning ▾

A single monolithic network causes representational overload for long-horizon problems with non-smooth reachable-set boundaries. We partition horizon K into windows — each with independent weights. Later-time frozen windows become boundary conditions for earlier windows.

V_θ(x,k) = b(x) if k=K | V^w_θ(x,k) if K−wW ≤ k < K−(w−1)W

↳Avoids monolithic representational overload
↳Frozen windows serve as stable BCs for earlier segments
↳Each network focuses capacity on a localized temporal segment

EQ.7 2 — Bang-Bang Probing ▾

Unlike PINNs, we infer optimal safety and disturbance actions via discrete value comparisons of forward-simulated next states — no analytic ∇ₓV needed. Only 2(n_u+n_d) evaluations per state convert unstable minimax RL into supervised regression.

û*_j = ū_j if C_{u,j} > 0 | u̲_j if C_{u,j} ≤ 0 C_{u,j} ≈ ΔV^u_j / Δt

↳No analytic gradient computation required
↳Converts minimax RL into stable supervised regression
↳Confidence weighting downweights flat-value regions

§3.2 3 — Mixed TD Targets ▾

50/50 blend of one-step teacher targets (Bellman-Isaacs anchor) and multi-step student rollout targets prevents compounding suboptimality. Time indices near the current boundary are oversampled to lock boundary-condition compliance.

y_mixed = 0.5 · y_teacher + 0.5 · y_student (M-step rollout)

↳Teacher targets preserve HJI recursion consistency
↳Student targets reduce policy drift over long rollouts

EQ.8 4 — Rollout Finetuning ▾

Before freezing each window, a finetuning phase calibrates the value network against empirical rollout targets under the current student policies. An asymmetric false-positive penalty suppresses states falsely predicted safe.

L_FT = L^student_TD + λ · Σ w_m(V_θ−G_m)² w_m=1+λ_fp·1{V>0, G<0}

↳Eliminates bootstrap drift across frozen window seams
↳Asymmetric penalty suppresses false-safe predictions

Figure 1 — Backward temporal curriculum with actor-critic learning

Method Overview: backward temporal curriculum with actor-critic learning

Left: Curriculum propagates backward from boundary condition (BC) at k=K. Each window trains, finetunes, then freezes as BC for the preceding window. Right: Within each step, the policy imitates bang-bang teacher probes and the value is updated with mixed TD(0)/TD(λ) targets.

// 03 — Interactive Sandbox

Live Physics Simulation

Toggle the HJ safety filter on/off to see the backward reachable tube in action. Click inside the canvas to re-anchor the initial state.

Live Simulation

Parameters

HJ Safety Filter

Active — prevents state from breaching BRT boundary.

Control Authority (ū) 1.8

Disturbance (d̄) 0.50

Gust Scale 50%

Simulation Speed

Show BRT overlay

Insight

The crimson BRT overlay marks states where worst-case disturbances exceed control authority — failure is mathematically inevitable without intervention.

Click canvas to re-anchor initial state

Diagnostics

Safety Value V(x,t) 0.8500

V<0 UnsafeBoundaryV>0 Safe

Control u_k

-0.245

Nominal

Disturbance d_k

+0.122

System noise

Status

Viable control strategy exists under worst-case disturbance. V(x) > 0.

Drone trajectory guided by LQR, protected by neural Bellman safety certificate.

// 04 — Results

Benchmark Comparisons

20K samples · 5 random seeds · FPR = dangerous over-optimism ↓ · FNR = over-conservatism ↓ · Succ ↑

Success Rate ↑

Ours 🏆99.65%

RL Baseline (ISAACS)3.70%

DeepReachMPC (PINN)2.50%

Method	FPR% ↓	FNR% ↓	Succ% ↑	Time(h) ↓
Ours	100† ± 0.00	0.00 ± 0.00	99.65 ± 0.31	4.0
DeepReachMPC	0.00 ± 0.00	100.0 ± 0.00	2.50 ± 0.90	5.0
RL Baseline	0.00 ± 0.00	93.40 ± 0.70	3.70 ± 0.72	5.5

† All evaluated initial states are inherently safe; 100% FPR reflects long-horizon (T=8s) policy execution errors, not value function over-optimism. The value correctly identifies states as safe — see §4.

Method	FPR% ↓	FNR% ↓	Succ% ↑	Time(h) ↓
Ours	0.50 ± 0.00	9.70 ± 1.00	0.50 ± 0.01	5.5
DeepReachMPC	2.00 ± 0.80	78.40 ± 14.00	0.06 ± 0.00	4.0
RL Baseline	30.38 ± 3.39	0.00 ± 0.00	0.00 ± 0.00	5.5

The 0.50% success rate reflects physical reality — most random initial states are dynamically doomed by gravity (z-velocity constraint). Our policy outperforms DeepReachMPC's built-in MPC solver (0.03%) and a carefully tuned MPPI controller (0.21%). RL Baseline FPR of 30.38% signals dangerous over-optimism.

System	Method	FPR% ↓	FNR% ↓	MSE ↓
40D Publisher-Subscriber Network
40D	Ours	0.000 ± 0.000	9.134 ± 0.693	0.009 ± 0.002
40D	DeepReachMPC	2.648 ± 0.310	0.001 ± 0.001	0.007 ± 0.001
40D	RL Baseline	23.306 ± 0.397	76.228 ± 0.602	1.717 ± 0.086
80D Publisher-Subscriber Network
80D	Ours	0.219 ± 0.429	14.263 ± 1.884	0.072 ± 0.016
80D	DeepReachMPC	2.115 ± 0.145	0.000 ± 0.000	0.026 ± 0.001
80D	RL Baseline	25.581 ± 0.941	78.696 ± 0.731	5.886 ± 0.248

Our generalized two-player zero-sum solver matches DeepReachMPC's MSE — despite DeepReachMPC being engineered for single-player optimal control. The RL baseline collapses entirely at 80D (MSE > 5.8, FNR > 78%).

Policy Objective	FPR% ↓	FNR% ↓	Succ% ↑	Time (min) ↓
Policy Gradient — 1 Step	99.58 ± 0.09	0.79 ± 0.74	92.20 ± 6.11	265.5
Policy Gradient — 3 Step	93.57 ± 8.71	0.25 ± 0.13	90.15 ± 6.87	307.4
Policy Gradient — 5 Step	99.75 ± 0.09	0.28 ± 0.83	80.81 ± 3.85	400.1
Teacher-Student (Ours)	100† ± 0.0	0.00 ± 0.00	99.65 ± 0.31	262.4

Teacher Mixing α	ΔSucc% ↓	ΔCost Mean ↓	ΔCost Std ↓
Pure Student (α=0.0)	0.2210 ± 0.0859	0.0356 ± 0.0015	0.0772 ± 0.0021
Pure Teacher (α=1.0)	0.3362 ± 0.0608	0.0274 ± 0.0013	0.0643 ± 0.0025
Decaying (α: 1→0)	0.2267 ± 0.0607	0.0299 ± 0.0070	0.0699 ± 0.0061
Constant α=0.5 (Ours)	0.2056 ± 0.0856	0.0312 ± 0.0018	0.0706 ± 0.0026

Method	False Positives ↓	False Negatives ↓
Ours	206	35
OptimizedDP	1,569	10

Pursuer Policy	Evader: MPPI	Evader: Ours
Ours	88%	42%
MPPI	36%	0%

Time Discretization Sensitivity · Narrow Passage · 20,000 test states

Step Size Δt	FPR% ↓	FNR% ↓	Succ% ↑	Time (h) ↓
Δt = 0.025s	100.0	0.0	82.01	11
Δt = 0.050s	100.0	0.0	94.29	5
Δt = 0.075s	100.0	0.0	93.49	4
Δt = 0.100s	100.0	0.0	96.31	3
Δt = 0.125s ★	100.0	0.0	99.99	2.5
Δt = 0.150s	100.0	0.0	99.98	1.5
Δt = 0.300s	0.0	100.0	78.75	1

Performance is robust across a wide range of Δt. Moderate steps (0.1–0.15s) achieve the best success rates with substantially reduced training time. Very small Δt (0.025s) expands the curriculum horizon, increasing bootstrap accumulation and training cost. Excessively coarse steps (0.3s) flip FNR to 100%, indicating the local dynamics approximation breaks down. ★ = used in main experiments.

// 05 — Video Demonstrations

Real-Time Testing

Side-by-side: HJ safety filter vs unfiltered nominal controller.

F1Tenth with Safety Filter

10 laps · zero collisions · avg 1.07 m/s on unseen physical track. All compute runs offboard over WiFi with ~50 ms round-trip.

✓ Filtered — 0 collisions

F1Tenth without Safety Filter

Unfiltered MPPI tracks racing line at max speed — 3 collisions within the very first lap on the physical track.

✗ Unfiltered — 3 crashes lap 1

// 06 — F1Tenth Sim-to-Real

Vision-Based Racing & Physical Deployment

BRT safety filter for high-speed autonomous racing (max 10 m/s). Model ingests a 128×128 egocentric BEV image + 5D proprioceptive state + time (16,390D total) + disturbance scale. At test time the learned value function acts as a Discrete-Time Control Barrier Function (DCBF) filter over a nominal MPPI controller.

0 / 50

Collisions with safety filter (simulation)

Avg speed 9.25 m/s (in-distribution) and 9.17 m/s (OOD tracks). Zero-shot to 3 held-out tracks under disturbance scale s_d = 0.1.

56% / 48%

Collision rate without filter (simulation)

Unfiltered MPPI on in-distribution and OOD tracks respectively. Failures at high slip angles near sharp turns. LiDAR insufficient — BEV is essential.

10 laps

Zero collisions — physical RC car (Rosmaster R2)

Avg 1.07 m/s at hardware limit (1.5 m/s). s_d = 0.2 absorbs ~50 ms WiFi latency + sim-to-real gap on a previously unseen track.

3 crashes

Unfiltered policy — first lap only

Nominal MPPI blindly tracks racing line at max speed, colliding within its very first lap on the physical track.

Hardware Setup

🚗

Platform

Rosmaster R2 RC scale car

📡

Localization

Vicon motion capture system

🔦

Onboard Sensor

7 Hz LiDAR (map construction)

💻

Offboard Compute

GTX 1060 · 12-core 3.3 GHz

📶

Communication

WiFi · ~50 ms round-trip

👁️

Input Dimensionality

128×128 BEV + 5D proprioceptive + time = 16,390D

// 07 — Citation

BibTeX

Anonymous BibTeX Reference

@inproceedings{anonymous2026gradfreereachability,
  title     = {Gradient-Free Neural Hamilton-Jacobi Reachability
               for Scalable Safety-Critical Control},
  author    = {Anonymous Author(s)},
  booktitle = {10th Conference on Robot Learning (CoRL)},
  year      = {2026},
  note      = {Submitted. Anonymous review version.}
}