Skip to content

ClipStepSizeController(jump_ts=...) skips time at every jump, and reads the wrong side of a jump at t1 #794

Open

Description

@charleswilmot

ClipStepSizeController(jump_ts=...) (or PIDController(jump_ts=...)) keeps the stages off a discontinuity
by stepping around it: a step ends one ULP before the jump, and the next one starts two ULPs after it. That
has two costs. My comment in #766 describes the same two problems for fixed steps. Here they are for adaptive
steps, with a fix that covers both cases.

Problem 1: the last step can read the wrong side of a jump at t1. diffeqsolve must reach t1, so the
last step ends exactly on it. If the input switches at t1, which a held input does at its last sample, the
last step's stages at c = 1 read the value that comes after t1. Listing t1 in jump_ts does not help
in float32: see #791.

Problem 2: time is skipped at every jump. The few ULPs around each jump are never integrated. In float64
this is negligible. In float32, with many jumps, it becomes the largest error of the solve.

Proposed fix: tell the step where the jumps are, and keep the steps whole. The controller would step
exactly onto a jump, as it already does for step_ts, and report whether the next step starts on a jump (today's
made_jump) and whether it ends on one. The solver clamps its stages' times two ULPs inside the step at those
ends only:

y1, y_error, dense_info, solver_state, result = solver.step(
    terms, t0, t1, y0, args, solver_state, jump_at_t0=jump_at_t0, jump_at_t1=jump_at_t1
)

# inside AbstractRungeKutta.step, where each stage's time is computed:
lo = jnp.where(jump_at_t0, after(t0), t0)   # after / before: two nextafter / two prevbefore
hi = jnp.where(jump_at_t1, before(t1), t1)
ti = jnp.clip(t0 + c_i * dt, lo, hi)

jump_at_t0 is today's made_jump, renamed to pair with jump_at_t1; FSAL needs nothing more, since a step
with jump_at_t0 already evaluates its first stage afresh. No time is skipped, no stage reads a jump instant,
and the same flags serve StepTo with jumps (#766).

Measured. The fix can be emulated today with a wrapper solver and step_ts. The controller then lands steps
exactly on the jumps, and the wrapper works out the two flags from the jump times:

class _Inside(dfx.AbstractTerm):
    """A term whose vector field is read at times clamped into [lo, hi]."""
    term: dfx.AbstractTerm
    lo: jax.Array
    hi: jax.Array

    def vf(self, t, y, args):
        return self.term.vf(jnp.clip(t, self.lo, self.hi), y, args)

    def contr(self, t0, t1, **kwargs):
        return self.term.contr(t0, t1, **kwargs)

    def prod(self, vf, control):
        return self.term.prod(vf, control)


class StagesAtJumps(dfx.AbstractAdaptiveSolver):
    solver: dfx.AbstractAdaptiveSolver
    jump_ts: jax.Array
    term_structure = dfx.AbstractTerm
    interpolation_cls = dfx.LocalLinearInterpolation

    def step(self, terms, t0, t1, y0, args, solver_state, made_jump):
        jump_at_t0 = made_jump | jnp.any(t0 == self.jump_ts)
        jump_at_t1 = jnp.any(t1 == self.jump_ts)
        inside = _Inside(terms, jnp.where(jump_at_t0, after(t0), t0), jnp.where(jump_at_t1, before(t1), t1))
        y1, y_error, _, state, result = self.solver.step(inside, t0, t1, y0, args, solver_state, jump_at_t0)
        return y1, y_error, dict(y0=y0, y1=y1), state, result

    # order, error_order, init, func: delegated to self.solver (see the reproducer)

# _Inside: a term whose vector field is read at times clamped into [lo, hi] (see the reproducer)
solver = StagesAtJumps(dfx.Tsit5(), jump_ts=ts[1:])
controller = dfx.ClipStepSizeController(dfx.PIDController(rtol=1e-6, atol=1e-6), step_ts=ts)

Test problem (the same as in #766). y' = -a y + u(t), y(0) = 0, a = 3. u is held at 1000 random
levels on [0, 1], so it switches every 1 ms, and once more at t1. Tsit5, PIDController(rtol=1e-6, atol=1e-6).

Every run is compared with the exact solution of the same problem, computed interval by interval. In float32
the switch times are themselves rounded to float32, so the float32 runs are compared with the exact solution
for those rounded times. Errors are relative to max |y|:

precision y at the 999 samples before t1 y(t1) dy(t1)/da steps
jump_ts=ts[1:-1] float64 1.2e-13 1.5e-4 6.8e-9 1021
jump_ts=ts[1:-1] float32 7.0e-5 1.5e-4 5.5e-5 1023
StagesAtJumps + step_ts=ts float64 1.3e-14 2.2e-15 6.8e-9 1001
StagesAtJumps + step_ts=ts float32 3.4e-7 2.0e-7 4.2e-7 1001

The bold cells show the two problems:

  • Problem 1 is the 1.5e-4 at t1, in both precisions.
  • Problem 2 is the float32 7.0e-5 and 5.5e-5, about 700 float32 ULPs of y here.

With the fix, every cell is at rounding level, about 3 ULPs of y in float32. It also takes one step per
interval, where stepping around the jumps takes 1021-1023. The 6.8e-9 in the dy/da column is the accuracy of
the finite-difference reference itself.

diffrax 0.7.2, equinox 0.13.8, jax / jaxlib 0.12.0.1 (0.11.2 gives the same numbers), CPU.

Reproducer
"""Jumps with an adaptive step size controller, two ways (pure JAX + diffrax; companion to diffrax issue #766).

The problem: y' = -a y + u(t), y(0) = 0, a = 3, where u is held at 1000 random levels on [0, 1] (it switches
every 1 ms, and once more at t1). Tsit5, PIDController(rtol=atol=1e-6). Two ways to keep the stages off the
switches:

  jump_ts        PIDController(..., jump_ts=ts[1:-1]): each step stops one ULP before a switch and resumes two
                 after it (ClipStepSizeController)
  StagesAtJumps  ClipStepSizeController(PIDController(...), step_ts=ts): steps land exactly on the switches; a
                 wrapper solver clamps the stages' times two ULPs inside a step whose start or end is a switch,
                 and evaluates its first stage afresh after one -- what flags `jump_at_t0` / `jump_at_t1` handed to
                 the step would do

Each is compared with the exact solution of the same problem, computed interval by interval. In float32 the
switch times are themselves rounded, so the float32 run is compared with the exact solution for those rounded
times. Printed, as errors relative to max |y|: y at every sample but the last, y at t1, dy(t1)/da; and the steps.
"""

import diffrax as dfx
import equinox.internal as eqxi
import jax
import jax.numpy as jnp
import numpy as np

jax.config.update("jax_enable_x64", True)

samples, a = 1000, 3.0
levels = np.random.default_rng(0).uniform(-1.0, 1.0, samples + 1)


def after(t):
    return eqxi.nextafter(eqxi.nextafter(t))


def before(t):
    return eqxi.prevbefore(eqxi.prevbefore(t))


class _Inside(dfx.AbstractTerm):
    """A term whose vector field is read at times clamped into [lo, hi]."""

    term: dfx.AbstractTerm
    lo: jax.Array
    hi: jax.Array

    def vf(self, t, y, args):
        return self.term.vf(jnp.clip(t, self.lo, self.hi), y, args)

    def contr(self, t0, t1, **kwargs):
        return self.term.contr(t0, t1, **kwargs)

    def prod(self, vf, control):
        return self.term.prod(vf, control)


class StagesAtJumps(dfx.AbstractAdaptiveSolver):
    """A solver told where the jumps are: a step that starts on one evaluates its first stage afresh, and the
    stages of a step that starts or ends on one are read two ULPs inside it. Emulates `jump_at_t0`/`jump_at_t1`."""

    solver: dfx.AbstractAdaptiveSolver
    jump_ts: jax.Array
    term_structure = dfx.AbstractTerm
    interpolation_cls = dfx.LocalLinearInterpolation  # unused: saves land on step ends

    def order(self, terms):
        return self.solver.order(terms)

    def error_order(self, terms):
        return self.solver.error_order(terms)

    def init(self, terms, t0, t1, y0, args):
        return self.solver.init(terms, t0, t1, y0, args)

    def step(self, terms, t0, t1, y0, args, solver_state, made_jump):
        jump_at_t0 = made_jump | jnp.any(t0 == self.jump_ts)
        jump_at_t1 = jnp.any(t1 == self.jump_ts)
        inside = _Inside(terms, jnp.where(jump_at_t0, after(t0), t0), jnp.where(jump_at_t1, before(t1), t1))
        y1, y_error, _, state, result = self.solver.step(inside, t0, t1, y0, args, solver_state, jump_at_t0)
        return y1, y_error, dict(y0=y0, y1=y1), state, result

    def func(self, terms, t0, y0, args):
        return self.solver.func(terms, t0, y0, args)


def grid(dtype):
    return jnp.linspace(0.0, 1.0, samples + 1, dtype=dtype), jnp.asarray(levels, dtype=dtype)


def solve(way, a, dtype):
    ts, us = grid(dtype)

    def field(t, y, args):
        u = us[jnp.clip(jnp.searchsorted(ts, t, side="right") - 1, 0, samples)]
        return -a * y + u

    pid = dfx.PIDController(rtol=1e-6, atol=1e-6)
    solver, controller = {
        "jump_ts": (dfx.Tsit5(), dfx.ClipStepSizeController(pid, jump_ts=ts[1:-1])),
        "StagesAtJumps": (StagesAtJumps(dfx.Tsit5(), jump_ts=ts[1:]), dfx.ClipStepSizeController(pid, step_ts=ts)),
    }[way]
    sol = dfx.diffeqsolve(dfx.ODETerm(field), solver, ts[0], ts[-1], 1e-4, jnp.asarray(0.0, dtype),
                          stepsize_controller=controller, saveat=dfx.SaveAt(ts=ts), max_steps=100_000)
    return sol.ys, sol.stats["num_steps"]


def exact(dtype, a):
    ts, us = (np.asarray(x, dtype=np.float64) for x in grid(dtype))
    y, out = 0.0, [0.0]
    for k in range(samples):
        h = ts[k + 1] - ts[k]
        y = us[k] / a + (y - us[k] / a) * np.exp(-a * h)
        out.append(y)
    return np.asarray(out)


for way in ("jump_ts", "StagesAtJumps"):
    for dtype in (jnp.float64, jnp.float32):
        reference = exact(dtype, a)
        eps = 1e-6
        gradient = (exact(dtype, a + eps)[-1] - exact(dtype, a - eps)[-1]) / (2 * eps)
        scale = np.max(np.abs(reference))
        with jax.enable_x64(dtype == jnp.float64):
            a_ = jnp.asarray(a, dtype)
            ys, steps = jax.jit(lambda a_: solve(way, a_, dtype))(a_)
            ys = np.asarray(ys, dtype=float)
            g = float(jax.grad(lambda a_: solve(way, a_, dtype)[0][-1])(a_))
        before_t1 = np.max(np.abs(ys[:-1] - reference[:-1])) / scale
        at_t1 = abs(ys[-1] - reference[-1]) / scale
        print(f"{way:14s} {jnp.dtype(dtype).name:8s} y before t1: {before_t1:.1e}   y(t1): {at_t1:.1e}   "
              f"dy(t1)/da: {abs(g - gradient) / abs(gradient):.1e}   steps: {int(steps)}")

Activity

  1. Shoryamishra61 commented on Nov 4, 2026

    @Shoryamishra61

    Hi @charleswilmot, thanks for the detailed reproduction and stage-aware approach.

    I've been investigating both failure modes locally on Diffrax main (e950aa2), with AI assistance and manual source review.

    I found that _bump_next_t0 skips the floating-point interval around each jump, while _clip_to_end can snap prevbefore(t1) back to t1, allowing final Runge–Kutta stages to evaluate the post-jump vector field.

    In my local diagnostic experiments, a terminal discontinuity increased Tsit5 from 5 steps to 117 (56 rejections). I also investigated a minimal workaround that preserves prevbefore(t1), but it causes problems with SaveAt(t1) and dense evaluation, so that approach does not appear sufficient.

    Before implementing a larger change, I'd appreciate maintainer guidance on the preferred architecture:

    1. Make Runge–Kutta stages explicitly aware of jump boundaries while preserving the full integration interval.
    2. Explore a per-step vector-field wrapper or another approach that avoids changing the public AbstractSolver.step interface.

    I haven't implemented or validated a production fix yet. I'd like to agree on the intended semantics and scope first, especially regarding custom solvers, autodifferentiation and compatibility with #773/#777.

    Happy to continue with a focused implementation once the direction is clear.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions