Repository navigation
Description
ClipStepSizeController(jump_ts=...) (or PIDController(jump_ts=...)) keeps the stages off a discontinuity
by stepping around it: a step ends one ULP before the jump, and the next one starts two ULPs after it. That
has two costs. My comment in #766 describes the same two problems for fixed steps. Here they are for adaptive
steps, with a fix that covers both cases.
Problem 1: the last step can read the wrong side of a jump at t1. diffeqsolve must reach t1, so the
last step ends exactly on it. If the input switches at t1, which a held input does at its last sample, the
last step's stages at c = 1 read the value that comes after t1. Listing t1 in jump_ts does not help
in float32: see #791.
Problem 2: time is skipped at every jump. The few ULPs around each jump are never integrated. In float64
this is negligible. In float32, with many jumps, it becomes the largest error of the solve.
Proposed fix: tell the step where the jumps are, and keep the steps whole. The controller would step
exactly onto a jump, as it already does for step_ts, and report whether the next step starts on a jump (today's
made_jump) and whether it ends on one. The solver clamps its stages' times two ULPs inside the step at those
ends only:
y1, y_error, dense_info, solver_state, result = solver.step(
terms, t0, t1, y0, args, solver_state, jump_at_t0=jump_at_t0, jump_at_t1=jump_at_t1
)
# inside AbstractRungeKutta.step, where each stage's time is computed:
lo = jnp.where(jump_at_t0, after(t0), t0) # after / before: two nextafter / two prevbefore
hi = jnp.where(jump_at_t1, before(t1), t1)
ti = jnp.clip(t0 + c_i * dt, lo, hi)jump_at_t0 is today's made_jump, renamed to pair with jump_at_t1; FSAL needs nothing more, since a step
with jump_at_t0 already evaluates its first stage afresh. No time is skipped, no stage reads a jump instant,
and the same flags serve StepTo with jumps (#766).
Measured. The fix can be emulated today with a wrapper solver and step_ts. The controller then lands steps
exactly on the jumps, and the wrapper works out the two flags from the jump times:
class _Inside(dfx.AbstractTerm):
"""A term whose vector field is read at times clamped into [lo, hi]."""
term: dfx.AbstractTerm
lo: jax.Array
hi: jax.Array
def vf(self, t, y, args):
return self.term.vf(jnp.clip(t, self.lo, self.hi), y, args)
def contr(self, t0, t1, **kwargs):
return self.term.contr(t0, t1, **kwargs)
def prod(self, vf, control):
return self.term.prod(vf, control)
class StagesAtJumps(dfx.AbstractAdaptiveSolver):
solver: dfx.AbstractAdaptiveSolver
jump_ts: jax.Array
term_structure = dfx.AbstractTerm
interpolation_cls = dfx.LocalLinearInterpolation
def step(self, terms, t0, t1, y0, args, solver_state, made_jump):
jump_at_t0 = made_jump | jnp.any(t0 == self.jump_ts)
jump_at_t1 = jnp.any(t1 == self.jump_ts)
inside = _Inside(terms, jnp.where(jump_at_t0, after(t0), t0), jnp.where(jump_at_t1, before(t1), t1))
y1, y_error, _, state, result = self.solver.step(inside, t0, t1, y0, args, solver_state, jump_at_t0)
return y1, y_error, dict(y0=y0, y1=y1), state, result
# order, error_order, init, func: delegated to self.solver (see the reproducer)
# _Inside: a term whose vector field is read at times clamped into [lo, hi] (see the reproducer)
solver = StagesAtJumps(dfx.Tsit5(), jump_ts=ts[1:])
controller = dfx.ClipStepSizeController(dfx.PIDController(rtol=1e-6, atol=1e-6), step_ts=ts)Test problem (the same as in #766). y' = -a y + u(t), y(0) = 0, a = 3. u is held at 1000 random
levels on [0, 1], so it switches every 1 ms, and once more at t1. Tsit5, PIDController(rtol=1e-6, atol=1e-6).
Every run is compared with the exact solution of the same problem, computed interval by interval. In float32
the switch times are themselves rounded to float32, so the float32 runs are compared with the exact solution
for those rounded times. Errors are relative to max |y|:
| precision | y at the 999 samples before t1 |
y(t1) |
dy(t1)/da |
steps | |
|---|---|---|---|---|---|
jump_ts=ts[1:-1] |
float64 | 1.2e-13 | 1.5e-4 | 6.8e-9 | 1021 |
jump_ts=ts[1:-1] |
float32 | 7.0e-5 | 1.5e-4 | 5.5e-5 | 1023 |
StagesAtJumps + step_ts=ts |
float64 | 1.3e-14 | 2.2e-15 | 6.8e-9 | 1001 |
StagesAtJumps + step_ts=ts |
float32 | 3.4e-7 | 2.0e-7 | 4.2e-7 | 1001 |
The bold cells show the two problems:
- Problem 1 is the 1.5e-4 at
t1, in both precisions. - Problem 2 is the float32 7.0e-5 and 5.5e-5, about 700 float32 ULPs of
yhere.
With the fix, every cell is at rounding level, about 3 ULPs of y in float32. It also takes one step per
interval, where stepping around the jumps takes 1021-1023. The 6.8e-9 in the dy/da column is the accuracy of
the finite-difference reference itself.
diffrax 0.7.2, equinox 0.13.8, jax / jaxlib 0.12.0.1 (0.11.2 gives the same numbers), CPU.
Reproducer
"""Jumps with an adaptive step size controller, two ways (pure JAX + diffrax; companion to diffrax issue #766).
The problem: y' = -a y + u(t), y(0) = 0, a = 3, where u is held at 1000 random levels on [0, 1] (it switches
every 1 ms, and once more at t1). Tsit5, PIDController(rtol=atol=1e-6). Two ways to keep the stages off the
switches:
jump_ts PIDController(..., jump_ts=ts[1:-1]): each step stops one ULP before a switch and resumes two
after it (ClipStepSizeController)
StagesAtJumps ClipStepSizeController(PIDController(...), step_ts=ts): steps land exactly on the switches; a
wrapper solver clamps the stages' times two ULPs inside a step whose start or end is a switch,
and evaluates its first stage afresh after one -- what flags `jump_at_t0` / `jump_at_t1` handed to
the step would do
Each is compared with the exact solution of the same problem, computed interval by interval. In float32 the
switch times are themselves rounded, so the float32 run is compared with the exact solution for those rounded
times. Printed, as errors relative to max |y|: y at every sample but the last, y at t1, dy(t1)/da; and the steps.
"""
import diffrax as dfx
import equinox.internal as eqxi
import jax
import jax.numpy as jnp
import numpy as np
jax.config.update("jax_enable_x64", True)
samples, a = 1000, 3.0
levels = np.random.default_rng(0).uniform(-1.0, 1.0, samples + 1)
def after(t):
return eqxi.nextafter(eqxi.nextafter(t))
def before(t):
return eqxi.prevbefore(eqxi.prevbefore(t))
class _Inside(dfx.AbstractTerm):
"""A term whose vector field is read at times clamped into [lo, hi]."""
term: dfx.AbstractTerm
lo: jax.Array
hi: jax.Array
def vf(self, t, y, args):
return self.term.vf(jnp.clip(t, self.lo, self.hi), y, args)
def contr(self, t0, t1, **kwargs):
return self.term.contr(t0, t1, **kwargs)
def prod(self, vf, control):
return self.term.prod(vf, control)
class StagesAtJumps(dfx.AbstractAdaptiveSolver):
"""A solver told where the jumps are: a step that starts on one evaluates its first stage afresh, and the
stages of a step that starts or ends on one are read two ULPs inside it. Emulates `jump_at_t0`/`jump_at_t1`."""
solver: dfx.AbstractAdaptiveSolver
jump_ts: jax.Array
term_structure = dfx.AbstractTerm
interpolation_cls = dfx.LocalLinearInterpolation # unused: saves land on step ends
def order(self, terms):
return self.solver.order(terms)
def error_order(self, terms):
return self.solver.error_order(terms)
def init(self, terms, t0, t1, y0, args):
return self.solver.init(terms, t0, t1, y0, args)
def step(self, terms, t0, t1, y0, args, solver_state, made_jump):
jump_at_t0 = made_jump | jnp.any(t0 == self.jump_ts)
jump_at_t1 = jnp.any(t1 == self.jump_ts)
inside = _Inside(terms, jnp.where(jump_at_t0, after(t0), t0), jnp.where(jump_at_t1, before(t1), t1))
y1, y_error, _, state, result = self.solver.step(inside, t0, t1, y0, args, solver_state, jump_at_t0)
return y1, y_error, dict(y0=y0, y1=y1), state, result
def func(self, terms, t0, y0, args):
return self.solver.func(terms, t0, y0, args)
def grid(dtype):
return jnp.linspace(0.0, 1.0, samples + 1, dtype=dtype), jnp.asarray(levels, dtype=dtype)
def solve(way, a, dtype):
ts, us = grid(dtype)
def field(t, y, args):
u = us[jnp.clip(jnp.searchsorted(ts, t, side="right") - 1, 0, samples)]
return -a * y + u
pid = dfx.PIDController(rtol=1e-6, atol=1e-6)
solver, controller = {
"jump_ts": (dfx.Tsit5(), dfx.ClipStepSizeController(pid, jump_ts=ts[1:-1])),
"StagesAtJumps": (StagesAtJumps(dfx.Tsit5(), jump_ts=ts[1:]), dfx.ClipStepSizeController(pid, step_ts=ts)),
}[way]
sol = dfx.diffeqsolve(dfx.ODETerm(field), solver, ts[0], ts[-1], 1e-4, jnp.asarray(0.0, dtype),
stepsize_controller=controller, saveat=dfx.SaveAt(ts=ts), max_steps=100_000)
return sol.ys, sol.stats["num_steps"]
def exact(dtype, a):
ts, us = (np.asarray(x, dtype=np.float64) for x in grid(dtype))
y, out = 0.0, [0.0]
for k in range(samples):
h = ts[k + 1] - ts[k]
y = us[k] / a + (y - us[k] / a) * np.exp(-a * h)
out.append(y)
return np.asarray(out)
for way in ("jump_ts", "StagesAtJumps"):
for dtype in (jnp.float64, jnp.float32):
reference = exact(dtype, a)
eps = 1e-6
gradient = (exact(dtype, a + eps)[-1] - exact(dtype, a - eps)[-1]) / (2 * eps)
scale = np.max(np.abs(reference))
with jax.enable_x64(dtype == jnp.float64):
a_ = jnp.asarray(a, dtype)
ys, steps = jax.jit(lambda a_: solve(way, a_, dtype))(a_)
ys = np.asarray(ys, dtype=float)
g = float(jax.grad(lambda a_: solve(way, a_, dtype)[0][-1])(a_))
before_t1 = np.max(np.abs(ys[:-1] - reference[:-1])) / scale
at_t1 = abs(ys[-1] - reference[-1]) / scale
print(f"{way:14s} {jnp.dtype(dtype).name:8s} y before t1: {before_t1:.1e} y(t1): {at_t1:.1e} "
f"dy(t1)/da: {abs(g - gradient) / abs(gradient):.1e} steps: {int(steps)}")
Activity
Shoryamishra61 commentedon Nov 4, 2026
Hi @charleswilmot, thanks for the detailed reproduction and stage-aware approach.
I've been investigating both failure modes locally on Diffrax main (
e950aa2), with AI assistance and manual source review.I found that
_bump_next_t0skips the floating-point interval around each jump, while_clip_to_endcan snapprevbefore(t1)back tot1, allowing final Runge–Kutta stages to evaluate the post-jump vector field.In my local diagnostic experiments, a terminal discontinuity increased Tsit5 from 5 steps to 117 (56 rejections). I also investigated a minimal workaround that preserves
prevbefore(t1), but it causes problems withSaveAt(t1)and dense evaluation, so that approach does not appear sufficient.Before implementing a larger change, I'd appreciate maintainer guidance on the preferred architecture:
AbstractSolver.stepinterface.I haven't implemented or validated a production fix yet. I'd like to agree on the intended semantics and scope first, especially regarding custom solvers, autodifferentiation and compatibility with #773/#777.
Happy to continue with a focused implementation once the direction is clear.