Close the eight findings from the expanded-catalog review - #18
Merged
Conversation
ProactiveClarification never applied the clarification answer. `stillMissing` was computed from the original triage only, so a user who supplied every slot was still told none were known and the booker "assumed" what it had just been told. The reply is now parsed per slot and merged through the gate. The merge guard is narrower than the review proposed, and the live run is why: rejecting every answer to a question that was not asked also discards a volunteered budget after the budget question was cut by the three-question cap, and the booker then invents a worse value. Volunteered slots merge; what the gate refuses is a reply silently rewriting a slot the request already settled. DualLlm implied taint made the extracted value safe. It does not. The injected EUR 48,000 is a well-formed decimal inside the range bound and would have filed - control flow intact, data flow corrupted. Added an unattended value limit that applies because a value is tainted, and the run now pushes the injected figure through both gates to show the type check passing and the policy refusing. Taint stops data becoming instructions; it does not make data true. EventDrivenAgents claimed a bounded channel and used CreateUnbounded. The bound is real but lives in the host counters, so the comment was the thing that was wrong. Also split TerminalEvents from DeadLetters with typed Refusal reasons - a workflow output filed as a delivery failure makes the queue useless as an alarm, which matters when this composes with AgentCommunicationFaultTolerance. MemoryPoisoningPrevention judged corroboration by trust class, which fails both ways: a scraper re-reading its seed page counted as independent, and two unrelated publishers could not corroborate at all. Source is now (Id, Trust) and independence is counted over evidence identity. MemoryConsolidation deleted its source episodes, so a slightly wrong summary became canonical and unfalsifiable. Episodes are archived with their ids recorded on the semantic memory; retrieval and ripeness filter to Active. GraphRAG lost provenance at the summariser: free-text summaries fed an answerer told to cite incident ids it might not have. Summaries are now structured, and the claimed source ids are checked against the graph rather than believed. ChainOfVerification called a same-model re-ask "independent measurement" and "verification wins". It is a blind cross-check: disagreement is strong evidence, agreement is weak. The verifier now signals CONFIDENT/UNCERTAIN, the reviser resolves into correct/contested/leave, and coverage is reported so a partially checked answer says so. LeastToMost carried answers forward as established facts with nothing checking them. Added a deterministic checkpoint where one exists - the billing schedule recomputed from the problem's own rules - with one retry and a contested outcome, and "[no verifier for this step]" everywhere else. Smaller: GraphOfThoughts enforces the six-sentence brief in host code rather than trusting the scorer's instruction; SpeculativeToolExecution drops the blanket 50% break-even rule; AgentRegistry documents that a verified card is discovery-time identity, not proof the peer controls it at connection time. 440 tests pass (30 new). All ten changed samples re-run against a live deployment; Pattern Explorer still serves 74 patterns. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0161UFxvL3zPhufoYaQh27Ss
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Applies every finding from the review of the expanded catalog, in the order it recommended. All eight were verified against the source before changing anything — each one was real.
The two that were actual bugs
1. ProactiveClarification never applied the clarification answer.
stillMissingwas computed from the original triage only, so a user who answered "Berlin, next Tuesday, 3 nights, max €150" was still told all four slots were unknown, and the booker "assumed" what it had just been told. The reply is now parsed per slot and merged through the gate.One deviation from the review, and the live run is why. The suggested rule — an answer may only fill a slot that was asked about — also discards a volunteered budget after the budget question was cut by the three-question cap, and the booker then invents €90 despite being told €150. Discarding information the user gave is the same failure the pattern exists to avoid, one step later. So volunteered slots merge; what the gate refuses is a reply silently rewriting a slot the request had already settled.
2. DualLlm implied taint made the value safe. It does not. The injected
48000.00is a well-formed decimal inside the range bound and would have filed — control flow intact, data flow corrupted. Added an unattended value limit that applies because a value is tainted, and the run now pushes the injected figure through both gates and prints the type check passing and the policy refusing:Modelling and taxonomy
3. EventDrivenAgents claimed a bounded channel and used
CreateUnbounded. The bound is real but lives in the host counters, so the comment was what was wrong — a bounded channel bounds in-flight events (backpressure) and its overflow modes block or silently drop, neither of which is what this needs. Also splitTerminalEventsfromDeadLetterswith typedRefusalreasons (NoSubscriber,GenerationLimit,RunBudgetExceeded).4. MemoryPoisoningPrevention judged corroboration by trust class, which fails in both directions: a scraper re-reading its seed page counted as independent, and two unrelated publishers could not corroborate at all.
Sourceis now(Id, Trust)and independence is counted over evidence identity. The demo now plants exactly that pair — same URL via a different mechanism (stays quarantined), then a contract record (promotes).5. MemoryConsolidation deleted its source episodes, so a slightly wrong summary became canonical and unfalsifiable. Episodes are archived with their ids recorded on the semantic memory; retrieval and ripeness filter to
Active.6. GraphRAG lost provenance at the summariser: free-text summaries fed an answerer told to cite incident ids it might not have. Summaries are structured, and claimed source ids are checked against the graph rather than believed — a fabricated id is reported and dropped.
Overclaims
7. ChainOfVerification called a same-model re-ask an "independent measurement" and told the reviser "verification wins". It is a blind cross-check: disagreement is strong evidence, agreement is weak, and a shared misconception survives both. The verifier now signals
CONFIDENT:/UNCERTAIN:, the reviser resolves into correct / contested / leave-alone, and coverage is reported so a partially checked answer says so instead of shipping under a "Verified answer" header.8. LeastToMost carried answers forward as established facts with nothing checking them. Added a deterministic checkpoint where one exists —
StepChecks.BillingTotalrecomputes the schedule from the problem's own rules, not a hardcoded 144 — with one retry and a contested outcome, and[no verifier for this step]everywhere else. The lesson is now externalising state makes validation possible, not externalised state is safer.Smaller
LengthPolicy) rather than trusting the scorer's instruction. Live run shows a 7-sentence draft capped 0.95 → 0.60 with its reason.Verification
Not done, per the review's own framing: no large refactor, and the observations on State Machine Agent, Control Plane, Mixture of Agents, Contrastive Explanation and Human-on-the-Loop needed no change.
🤖 Generated with Claude Code
https://claude.ai/code/session_0161UFxvL3zPhufoYaQh27Ss