Repository navigation
research(robotics): preregister paired fault-recovery benchmark - #2860
Draft
Tristan-Stoltz-ERC wants to merge 2 commits into
Draft
Tristan-Stoltz-ERC wants to merge 2 commits into
Tristan-Stoltz-ERC wants to merge 2 commits into
Conversation
This was referenced Oct 29, 2026
Tristan-Stoltz-ERC
force-pushed
the
research/outcome-lifecycle-ledger-v0-1
branch
from
October 29, 2026 13:18
8bb0454 to
3bb0641
Compare
Tristan-Stoltz-ERC
force-pushed
the
research/paired-robotics-recovery-benchmark-v0-1
branch
from
October 29, 2026 13:18
31ffb25 to
11937c3
Compare
This was referenced Oct 29, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Stacked on #2852. Adds a preregistered, simulation-first experiment to measure whether validated controller-outcome feedback improves recovery without weakening the controller authority boundary.
Added
docs/research/ROBOTICS_PAIRED_RECOVERY_BENCHMARK_V0_1.md: four-arm design (native-controller reference, Symthaea intent, guarded adapter, guarded feedback loop); six fault families; non-compensatory lifecycle/dispatch gates; paired statistics; evidence tiers and claim limits.docs/research/robotics-paired-recovery-scenarios-v0.1.json: frozen 30 nominal seed cases and 6 × 3 × 30 = 540 fault cases, for 570 paired scenario identities and 2,280 arm-episode records across four arms. Numeric values/timing that depend on the controller profile must be frozen before scored runs.Research hypothesis (not a result)
On the same scenario/seed, B3 should improve fault-recovery success against B2 while keeping nominal success within a preregistered 1 percentage-point non-inferiority margin. No weighted reward can compensate for a hard boundary violation. A proposed +10 percentage-point recovery improvement is a target, not a result or industry standard.
Claim ceiling
This PR defines the experiment; it does not implement a rollout runner, restart-safe persistence, a physical robot adapter, or a safety certification. The current in-memory ledger does not by itself provide durable restart fencing. Any missing, crashed, timed-out, or unresolved episode must be represented, not discarded.
Validation
Related: #2828, #2836, #2847, #2852.