[mmmries]

Summary

J5a of #1110: a JOIN transform (behind TRELLIS_DEV_GROUP_BASIS=1) now handles a TRUNCATE of either side and recomputes cells, where before it halted with GroupBasisUnsupported. The cell recompute covers image-less changes, quarantine releases and the automatic retry. Column quarantine, schema change and retyping, and the row-security and capture audits are J5b and are not in this PR.

Design

Per-side TRUNCATE (ADR-0019, "TRUNCATE of either side excludes writers of both"). The truncate's page takes the pair's lock row FOR UPDATE; every other writer takes it FOR SHARE. The page then clears what the truncate empties before it applies its other changes:

• TRUNCATE a: empties P and T.
• TRUNCATE b: empties M, sets every J.matches to 0, empties T, then refills T's staging from P (P ⊗ U for LEFT, nothing for INNER). The refill reads P before the page's own sides apply, so it composes with them.

Each side raises its own truncate floor, keyed by its hidden table, so a change from before the truncate is refused. The visible target is cleared through the seam with clear_target. J rows are kept.

The hidden tables are emptied with DELETE, not TRUNCATE. A release isn't a page, so the drain barrier doesn't keep it out: it can run alongside a truncate page. Before it reaches the lock row, it already holds ACCESS SHARE on the hidden tables from its working tables' create … as select … with no data. Parse analysis takes that lock even when the temp table already exists. A TRUNCATE's ACCESS EXCLUSIVE would then deadlock against it. The cost is that a TRUNCATE of a side deletes P (or M) and T row by row: WAL per row, and dead tuples left for autovacuum. That's the same cost the visible target's clear already pays. It's proportional to the cell count, not the source's row count, but on a shape whose cells are near the fact table's size, one truncate page is large.

Two-sided barrier. The engine's drain barrier is already stronger than ADR-0019 requires. A truncate-bearing segment drains alone, after every earlier segment of any table and before every later one, and is never coalesced. That covers mixed pages, and no new mechanism was needed. family10_a_mixed_page_waits_for_an_earlier_truncate pins it.

Cell recompute. A JOIN side's GroupTargetPlan now has a table of the cells to recompute and its source's row identity. An image-less change names its key's current row's cell and its prior-image hint's cell; for M, cells with a NULL key part are filtered out. These cells go through the J-locked writer, in this order:

1. The lock row.
2. J inserted for the named keys, then J locked sorted.
3. The cells locked sorted. The existing ADR-0002 recompute statement then rewrites each cell from its source with the snapshot as its basis, and records each cell's change in the side's deltas table.
4. The product step sums the deltas per cell (a page may both apply and recompute one cell) and carries the change into T and J.matches (JR1–JR3).

Release and automatic retry. Parking a JOIN key records the cells of its side in agg_groups. release_key (and so retry_held_key) calls join::release, which recomputes those cells and the key's current row's cell in the release transaction, using the same steps.

No migration.

Verification

New tests in trellis/tests/join_interleavings.rs are deterministic and don't poll:

• J1 family 10, as races between a truncate page and another page:
  • TRUNCATE a/b vs. a page of the other side.
  • A page of the same side vs. its truncate.
  • Truncate + insert in one transaction.
  • Both batch orders, and each page frozen at all 7 points; the other page must queue on the lock row.
  • Run on LEFT, INNER and the pair key.
• Family 10, sequential, on all six kinds: both sides truncated in either order, then refilled. TRUNCATE b with every key matched leaves all J.matches at 0, keeps every J row, empties the INNER targets and leaves only NULL-gb groups in LEFT.
• Family 10: a stale page from before the truncate, drained out of order after it, is refused by the floor.
• The mixed-page barrier (S10.mixed-page-barrier): the mixed batch can't be claimed until the earlier truncate's batch drains.
• An image-less change recomputes the cells it names on either side: it repairs changes made with the capture triggers off, including a deleted row named only by its prior-image hint, and a flip to unmatched.
• A page that both applies captured changes to a cell and recomputes it, on either side, including a flip to unmatched hidden behind a captured insert and delete in the same page. The cell gets two delta rows, and the product must sum them.
• A TRUNCATE of the joined side refills unmatched groups whose joined-side GROUP BY columns are timestamptz and numeric(10,2), on a text/integer pair key, with a NULL-key fact. The refill's bare nulls take the target columns' types from INSERT … SELECT.
• A release recomputes every cell the held key could owe, on either side, on all six kinds. The test asserts the held cells and checks the targets right after the release, before anything drains, so the release's own recompute must repair them, not the Recompute it stages. The automatic-retry test checks the same way.
• A release is a J-locked writer: frozen at each point, a page of the other side on the same key queues behind it exactly from its J lock on.

Mutation checks, each run against these tests and then reverted:

• Skipping the recompute fails the 4 recompute, release and retry tests. Making join::release a no-op fails the 3 release and retry tests.
• Taking any one delta row per cell instead of their sum fails the apply-and-recompute test.
• Taking the lock row FOR SHARE on a truncate page fails the family 10 races: a page of the other side ran during the frozen truncate and left T ≠ Σ P ⊗ M.
• Skipping the refill fails the LEFT truncate tests and the typed-refill test.

join_interleavings.rs takes about 70 s (40 tests).

verify: green, passed=3593 failed=0 ignored=40 (tree 09593a3d50, rebased on main e3e08e4)

Part of #1110

🤖 Generated with Claude Code