perf(review): retire dry chunks and pipeline verification in the reverse audit - #8498
perf(review): retire dry chunks and pipeline verification in the reverse audit#8498wenshao wants to merge 3 commits into
Conversation
|
🔄 Qwen Triage is running — watch live progress. Stage results will post in this thread as they complete. 🔄 Qwen Triage 正在运行 —— 查看实时进度。各阶段结果完成后会更新在本线程。 |
|
Re-run over the four commits landed since the last pass (
Moving on to code review of the delta. 🔍 中文说明针对上次评审之后落地的四个提交（
进入增量的代码审查。🔍 — Qwen Code · qwen3.8-max Reviewed at |
Code review — the delta since
|
| Check | Conclusion |
|---|---|
Classify PR |
✅ success |
Desktop Shell (ubuntu-22.04) |
✅ success |
Post Coverage Comment (ubuntu-latest, 22.x) |
✅ success |
Remind on force-push |
✅ success |
Test (ubuntu-latest, Node 22.x) |
✅ success |
authorize |
✅ success |
label |
✅ success |
review-config |
✅ success |
web-shell E2E Smoke (ubuntu-latest, Node 22.x) |
✅ success |
One row per check name (latest run); skipped checks omitted; failures sort first. / 每个检查名一行（取最新一次运行），省略 skipped，失败项排在最前。
On the sandboxed lanes: the scheduling mechanics — the behavioural half of this PR — are substantiated by 2b itself. Every behaviour the four fixes introduce is pinned by a test carrying the original probe's shape; revert any fix and its test fails. This triage run's verify job is in flight alongside this comment (its report will land in the verification thread) and can add a sandboxed execution signal, but it cannot settle the one remaining claim — the wall-clock projection — which takes a real multi-round review with live agents. No TUI surface to drive, so no tmux capture applies.
中文说明
代码审查——自 fc25e277 以来的增量
本次覆盖上次评审后落地的四个提交：51f0baf、d0ef34b（两轮 autofix），以及 2496f0e、4d64531（S1–S4 跟进，加上其后内部评审发现的产出探测器配对问题）。读 diff 前我的独立基线：注入性修复应当统计每份 transcript 匹配的 record 数而非反向，且不能破坏 whiff 重启的合并；--chunk 豁免需要一个被拒轮次不可能持有的磁盘键（准入 stamp）；record 围栏应与 transcript 同用 plan mtime 语义，而 coverage 调用方刻意不设围栏。四处修复都落在这个形状上——且每一处都带着要求它的那个探针转化成的测试。
已核验（新 head）：
- 注入性守卫现为"每份 transcript 对应的 record 数"（评审 2 的 Critical）：只有恰好匹配一条 record 的 transcript 能认证任何东西；一个拿到整轮拼接内容的 agent 不指名任何领地、无法让任何 chunk 退休。诚实形态双向不受影响：发射 prompt 内含轮号与领地，诚实发射恰好匹配自己的记录；同一条 record 的多份诚实 transcript（强制 whiff 重启）各自认证、再由合并裁决。双向均有测试钉死。
- 预算状态以 plan mtime 围栏（评审 1 的 Critical 1）：stamp 携带
planMtimeMs，读取按围栏过滤；别次 plan mtime 下写出的 stop marker 读作无。CI 重试不再被死 attempt 的拒绝封顶、也不再从死 attempt 的 stamp 量时间。无标签构建同样去重。双向测试齐备。 --chunk豁免改为按 stamp（S1）：持有准入 stamp 的轮次是修复、全豁免；无 stamp 的轮次经--chunk进入时与--all-chunks同一序列——先收敛、再预算、再盖章——首个 chunk 构建即该轮准入。三条准入路径共用admitReverseAuditRound，不会漂移。探针形状逐条钉死。- prompt record 同享围栏（S2）：调度器传入 plan mtime，与 transcript、预算文件对称；死 attempt 的陈旧孪生记录不再触发注入性守卫把退休静默关掉。不对称是刻意的：
coverage.ts两处调用不设围栏（义务不因过期而减免——已在 head 核实两处调用点）。 - 干净回执分类器按结构读取、双语（S3）：120 字符下限已移除——措辞（英文变体 +
未发现问题族中文）、中词连字符无法伪造的分隔符、其后必须点名对象（反引号、真实路径、≥4 个汉字或约 20 个扁平字符）。dry仍要求successfulToolCalls > 0 && diffToolCalls > 0。双向钉死，含"光杆套话双语都不算干净"。 - 累计清单给未验证条目打标签（S4，SKILL 内）：
— [unverified]在准入合并时添加、确认时移除、否决时删除条目；Step 6 只读无标签者——结构化而非靠记忆——循环结束时仍带标签意味着 verifier 失败：仅终端可见、绝不视为已确认。标签在 findings 文件内，随发射 prompt 折叠并进 record key 哈希，投递检查不受影响。这也吸收了评审 2 的残余担忧：未验证发现不再可能仅凭记忆失误泄入报告。 - 收敛先于预算闸、全局奇偶：
CONVERGED退出 5，无记录、无 stamp、无 marker——完成的审计不被封顶为截断。冷查落在全局偶数轮，错位的证书重新对齐，全退休出口在产出网格形状上依然可达。均钉死。 - 小项：round 1 估算上限为剩余预算三分之一（
/review --timeout=30不再拒绝一切 PR 的第 1 轮）；compose-review按 marker 的精确条目去重；transcript 先按角色标记预过滤、每份只扁平化一次；产出探测器要求 severity+file 成对，引用累计条目的 auditor 不再把自己的 chunk 钉成热区；表头截断自检描述了退休注记；converged要求非空 chunk 列表。
未发现阻塞——增量就是四处修复加各自测试，别无其他。残余问题，逐一点名而非隐藏：
- 流水线化后末轮验证先于 Step 6 的顺序仍是文案；标签让泄漏防护结构化，但这条顺序本身仍依赖编排者遵守。评审 2 建议的"按轮号键控 verify 记录"未被采纳——标签是更省的缓解，此处仍是唯一假设编排者合规的位置。
- 墙钟折算（95 → 约 55 分钟）是按实测轮成本的算术，其验证场是下一次带预算的大 PR CI 评审。
- 合并顺序：叠在仍未合并的 fix(review): stop the reverse-audit loop while there is still time to report #8468 上——需先合并那个，本 diff 随之缩为自己的提交。
测试证据——PR 自己的 CI
已评审 head 的 CI 全绿：Qwen Code CI（event pull_request）结论 success——ubuntu 单测、Desktop Shell、web-shell E2E smoke 全部通过。macOS/Windows/集成为 skipped，是 workflow 自身 ci_profile 门控（本 PR 分类未达 full 档），属仓库既有机制，不是失败信号。作者报告的套件数字（src/commands/review：1746 通过 / 0 失败）由此有了绿色 ubuntu 套件背书，不再只是说法。
（CI 状态表见上方英文区域，由 finalize workflow 维护。）
关于沙盒通道：调度机制——本 PR 的行为性部分——已由 2b 本身证实：四处修复引入的每个行为都由携带原探针形状的测试钉死，回退任何一处修复其测试即失败。本次 triage 运行的 verify 任务与本评论同时进行（报告将落在验证帖），可补充沙盒执行信号；但剩余的唯一主张——墙钟折算——需要真实多轮评审与活的 agent，任何沙盒通道都无法复现。无 TUI 表面，不适用 tmux 抓取。
— Qwen Code · qwen3.8-max
Reviewed at b2c181a367ade819fb652be2019a0fd0a8f3245e · re-run with @qwen-code /triage
|
Confidence: 4/5 — every finding from three probe-backed review rounds is fixed at the head, each fix pinned by a test carrying the original probe's shape, and CI is green on the reviewed commit; the remaining quarter is what reading alone cannot settle — the wall-clock arithmetic, and one ordering still held by prose. Stepping back over the whole arc: this is what iteration under real scrutiny looks like. The previous pass approved the mechanism at The two reservations, both named rather than blocking: the pipelined last-round verification ordering is still an instruction in the SKILL — the If I were maintaining this in six months, the comments would still save me — every fence, every exemption, every parity choice carries the measurement or the probe it exists for, in the established voice of the subsystem. Every change in the delta is load-bearing. Approving, pinned to the reviewed commit. 中文说明信心：4/5 —— 三轮带探针评审中的每一条发现都已在新 head 上修复，每处修复都由携带原探针形状的测试钉死，已评审提交上的 CI 全绿；剩下的一分是纯阅读无法落定的部分——墙钟折算，以及一处仍靠文案维持的顺序。 退一步看整条弧线：这是在真实审视下迭代的样子。上一轮在 两点保留意见，点名而不阻塞：流水线化后末轮验证的顺序仍是 SKILL 里的一条指令—— 若六个月后由我维护，注释依然会救我一命——每道围栏、每个豁免、每次奇偶选择都带着它赖以存在的测量或探针，用的是这个子系统既有的口吻。增量中每一处改动都承重。予以批准，锚定在已评审的提交上。 — Qwen Code · qwen3.8-max Reviewed at |
Review:
|
chunk A (dry 1,2 → lastDry=2) |
chunk B (yielded r1, dry 2,3 → lastDry=3) |
|
|---|---|---|
| round 4 | cold check (due) | skipped |
| round 5 | skipped | cold check (due) |
So on any diff where one chunk yields in round 1 or 2 — the common case on a large PR, and the exact case the PR's own yield grid shows — the loop never takes the clean-convergence exit and always runs to the 5-round cap. Combined with the SKILL edit that re-scopes the two-dry-rounds rule to 3A only ("the 3A criterion — one auditor, so round-dry and territory-dry are the same thing"), the 3B path is left with no early exit at all except exit 4/the cap, and reports "hard cap" rather than "converged" on a run that had in fact converged.
The rounds are cheaper, so this is not a perf regression — but it does mean the advertised exit-5 path will rarely fire in practice. Anchoring the parity on a single per-round basis (e.g. round % 2 against a fixed epoch, or "due if it was skipped last round") would let staggered chunks re-align.
🟡 Suggestion 2 — pipelining lets a rejected finding suppress one round of rediscovery
The cumulative list now carries unverified entries, and the auditor brief tells each agent not to re-report what is on it. If round k files a false positive at x.ts:20, round k+1's auditor is told that spot is covered; the verifier then rejects it and it leaves the list at the next merge — but round k+1 is already gone. Under the old ordering a rejected entry never entered the list at all.
The window is one round in one location and the cap still bounds the loop, so I don't think this blocks the change — but "with detection semantics intact" in the PR body overstates it, and the SKILL bullet would be more honest if it named the trade (an entry a verifier later rejects will have suppressed one round of rediscovery in its neighbourhood).
🟡 Suggestion 3 — the budget gate also refuses a single-chunk repair rebuild
agent-prompt.ts:1690 gates on role === 'reverse-audit' without distinguishing "admit a new round" from "rebuild one chunk of a round already admitted". So a --chunk <id> --round k repair after a truncated read gets exit 4 and writes a marker reading stopped before round k — even though round k ran. Two effects: the truncated delivery can't be repaired (that chunk's auditor is never launched, and nothing writes an unreviewedDimensions entry for it), and the disclosure is factually wrong about which round stopped. Retirement already exempts the --chunk path as "a repair is not scheduling" (nice test, by the way) — the same reasoning applies to the budget gate.
Also: for a rebuild of round k, expectedRoundSeconds deliberately reaches past round k's own stamp to round k-1's, so the "estimate" includes all of round k's elapsed time — systematically inflated exactly on the path most likely to be refused.
🟡 Suggestion 4 — the header's truncation self-check no longer describes the output
agent-prompt.ts:1526 still tells the reader "the output ends with an end-of-round line — if either is missing, the output was truncated in transit", but with a retirement the retirement: note is now appended after it. The literal test still works ("if either is missing"), but the sentence is the thing that has to survive a compression, and it now describes a shape the builder no longer emits. Worth a clause: …ends with an end-of-round line (followed by the retirement note, when there is one).
Smaller notes
--all-chunkswith no--round:stampRound(planPath, undefined)skips the dedupe (round !== undefined && …) and appends a{round: null}stamp on every invocation, andexpectedRoundSeconds(plan, undefined)then reads the most recent one. Harmless as long as the SKILL always passes--round, but it is a fail-closed corner in a module that is otherwise uniformly fail-open.DRY_RECEIPT_RE = /No (new )?(issues|findings|gap)/iwon't match a substantive receipt phrased "Nothing further to report — re-walked…". That fails toward auditing, so it's safe; just noting the heuristic is narrower than the brief's contract implies.- Workflow: with
attempt_timeoutas low as 30s (the loop's floor beforefail), the 600s reserve floor means the audit loop is skipped outright. The comment says that's intended ("degrades to skipping the audit loop, correctly") — agreed, worth keeping the comment. - The
3600↔DEFAULT_RESERVE_SECONDS"keep the two in sync" pairing is a comment-only invariant across a YAML file and a TS file. Nothing will catch a drift. Low stakes (both are fallbacks), but a# see deadline.tson the TS side too would at least make it bidirectional.
Test coverage
Strong where it exists — driving agentPromptCommand.handler round by round with harness-shaped transcripts is much better than unit-testing scheduleReverseAuditRound directly, and the negative assertions (no records, no stamp, exit 4 outranks exit 5) are the right ones. Gaps:
- No test for a schedule with mixed parity across chunks (Suggestion 1) — the scenario the yield grid in the PR description actually shows.
- No test for a second run against the same plan path (Critical 1). The retirement path is safe there by inheritance from the transcript fence;
deadline.tsis not, and nothing covers it. - No test for one transcript matching several records (Critical 2).
.github/workflows/qwen-code-pr-review.yml's new shell arithmetic (floor/cap, per-attempt recompute) has no coverage; the existing extract-the-run-step harness pattern would cover it cheaply.
Security / performance
No new attack surface: no path comes from the model (both deadline.ts and retirement.ts derive from planPath / the CLI-exported env, consistent with the rest of the module), no new network or shell. readTranscripts is now called once more per --all-chunks round from round 3 — an O(transcripts) parse of a directory that is already read at Step 3D and Step 6, so negligible against a round that costs tens of minutes. The projected saving looks plausible to me on the grid shown; the pipelining half is the more reliable of the two, since it doesn't depend on chunks going cold.
中文说明
总评
方向正确,工程质量高:退休调度由 CLI 强制而非 prose 要求;调度依据是 orchestrator 无法伪造的两份证物(自身的 prompt 记录 × harness transcripts);所有分类路径都朝"多审计"失败;readTranscripts 的 plan-mtime 围栏被正确继承。budget-stop marker 由 compose-review 直接读取、而不依赖 orchestrator 转述,是 verdict cap 的正确形状。9 个调度用例逐轮驱动真实 handler,而非孤立单测 scheduler,这是它们真正有价值的原因。
本地复现:未跑全量 src/commands/review,只跑了本 diff 触及的两个文件 deadline.test.ts(15)+ agent-prompt.test.ts(166),181 passed / 0 failed。两个 Critical 均在干净 worktree 上以探针复现。
🔴 Critical 1:预算状态会带入 CI 同 job 的重试并污染它
budget-rounds.json / budget-stop.json 放在 promptRecordDir(planPath),仅由 plan 路径推导。与 retirement.ts:173 正确使用 plan mtime 围栏的 readTranscripts 不同,这两个文件没有任何围栏,且新一次 capture 不会清理记录目录(promptRecordDir 的三个非测试调用方都不删除)。本该清理的 Step 9 恰恰在 attempt 提前死亡时不会执行,而 workflow 的重试就在同一个 job、同一个 workspace、同一个 plan 路径上。
后果两条,均已复现:
(a) 陈旧 marker 让健康的 attempt 2 被误封顶——compose-review.ts:350 无条件读取,attempt 2 即使完整跑完所有轮次,也会披露 reverse audit — stopped before round 3 by the review time budget 并把 Approve 压到 COMMENT。
(b) stampRound 按 round 去重,导致 attempt 2 根本无法打戳,于是每个估算都是从 attempt 1 的墙钟时间量出来的:round-2 估算 2160s(真实约 1500s)→ 在还剩 54 分钟、本轮只要 25 分钟时被拒。
即:大 PR 上一次瞬时 API 错误,可能让重试完全没有反审,还附带一条它从未触发过的时间预算披露——恰好把这道闸的意图反转。本地对一个上次被中断(未走 Step 9)的目标重新 review 同理。
建议:给这两个文件加上 transcripts 已有的同一道围栏(把 planMtimeMs 写进去、不匹配即丢弃,readRoundStamps / readBudgetStop 各几行),或让 capture 步骤清理 promptRecordDir。前者更省,改动不出 deadline.ts。
🔴 Critical 2:一份 transcript 可以让所有 chunk 退休
retirement.ts:194 会把每一条 prompt 出现在该 transcript 里的记录都算上,且 wasDeliveredVerbatim 是"可加不可删"的判定——所以一个把 N 个 block 拼接起来的 launch prompt 同时匹配 N 条记录。verificationGaps 正是防这一手的,注释写得很清楚:"Injective: one transcript may satisfy ONE roster requirement…"
对真实 scheduler 复现:两个 agent(每轮一个,拿到整轮输出)让 6 个 chunk-round 全部退休,round 3 直接 converged: true → exit 5 / CONVERGED,而那段 stderr 明确宣称"这是干净收敛,不欠 unreviewedDimensions"。verificationGaps 的 floor-of-one 也照样通过(那一个 agent 确实读了 brief),下游无人能拦。
这是 orchestrator 被观测到真做过的偷懒(injectivity 规则就是因此存在),不是假想。按本模块自述的威胁模型,退休判定应当采用与 verificationGaps 相同的"一份 transcript 只能满足一条记录"的贪心分配。
🟡 建议 1:CONVERGED 只在所有 chunk 的末次干净轮同奇偶时才可能触发
converged 要求所有退休 chunk 本轮都不到期,即所有 chunk 的 (round - lastDry) 同为奇数。两个 lastDry 奇偶不同的 chunk 因此永远无法在同一轮同时被跳过(A: dry 1,2;B: r1 出货、dry 2,3 → 第 4 轮 A 冷查 B 跳过,第 5 轮 A 跳过 B 冷查)。于是只要有一个 chunk 在第 1、2 轮出货(大 PR 的常态,也正是 PR 描述里那张网格的形状),循环就永远走不到干净收敛出口,只能撞 5 轮上限并报告"硬上限"而非"已收敛"。
轮次本身变小了,所以不是性能回退;但配合 SKILL 把两干净轮规则改写为仅限 3A,3B 路径实际上除 exit 4 与硬上限外再无提前出口。把奇偶锚定改成全局(如 round % 2 配固定纪元,或"上一轮被跳过则本轮到期")可让错位的 chunk 重新对齐。
🟡 建议 2:流水线化会让被否决的发现压制一轮复查
累计清单现在带未验证条目,而 auditor brief 要求不重复报告清单内容。若第 k 轮报了一条 x.ts:20 的误报,第 k+1 轮的 auditor 被告知该处已覆盖;verifier 随后否决、下次合并时移出——但第 k+1 轮已经过去了。旧顺序下被否决条目根本不会进入清单。
窗口只有一轮一处,上限仍然兜底,我不认为这该挡住改动;但 PR 描述里"检测语义不变"说过头了,SKILL 那条 bullet 若能点名这个代价会更诚实。
🟡 建议 3:预算闸同样会拒绝单 chunk 的修复重建
agent-prompt.ts:1690 只按 role === 'reverse-audit' 判断,不区分"准入新一轮"与"重建已准入轮次的某个 chunk"。于是截断后的 --chunk <id> --round k 修复会得到 exit 4,并写下一条 stopped before round k 的 marker——而第 k 轮其实跑过了。两个后果:截断的投递无法修复(该 chunk 的 auditor 永不发射,也没有任何 unreviewedDimensions 条目),且披露文本关于"哪一轮停止"是错的。退休逻辑已经把 --chunk 路径豁免为"修复不是调度"(那个用例写得很好),同样的理由适用于预算闸。
另外:重建第 k 轮时 expectedRoundSeconds 会刻意越过第 k 轮自己的戳去读第 k-1 轮的,于是"估算"包含了第 k 轮已耗时间——恰好在最可能被拒的路径上系统性偏高。
🟡 建议 4:表头的截断自检已经不描述实际输出
agent-prompt.ts:1526 仍写着"输出以 end-of-round 行结尾",但有退休时 retirement: 注记被追加在其后。字面判据("若缺失")仍成立,但这句话是要挺过压缩的那句,现在描述的形状 builder 已不再产出。建议补一个从句。
其他小点
--all-chunks不带--round时,stampRound(planPath, undefined)跳过去重、每次调用追加一个{round: null}戳,expectedRoundSeconds(plan, undefined)随后读到最近那个。只要 SKILL 始终传--round就无害,但这是一个整体 fail-open 的模块里唯一 fail-closed 的角落。DRY_RECEIPT_RE匹配不到"Nothing further to report — re-walked…"这类实质回执。朝多审计失败,安全;只是比 brief 契约更窄。- workflow:
attempt_timeout低至 30s 时,600s 保留下限会直接跳过整个反审循环——注释说这是预期行为,同意,值得保留这段注释。 3600↔DEFAULT_RESERVE_SECONDS的"保持同步"是一条跨 YAML/TS 的纯注释不变量,无人能捕获漂移。风险低(两者都是兜底),但 TS 侧也加一句反向指引会更好。
测试覆盖
已有部分很扎实——用 harness 形状的 transcript 逐轮驱动真实 handler,远好于直接单测 scheduleReverseAuditRound;负向断言(无记录、无戳、exit 4 优先于 exit 5)选得对。缺口:
- 无 chunk 间奇偶错位的调度用例(建议 1)——正是 PR 描述那张网格的形状。
- 无同一 plan 路径第二次运行的用例(Critical 1)。退休路径靠继承 transcript 围栏是安全的,
deadline.ts不是,且无人覆盖。 - 无一份 transcript 匹配多条记录的用例(Critical 2)。
- workflow 新增的 shell 算术(下限/上限、按 attempt 重算)无覆盖;现有的"抽出 run 步骤跑 fixture"手法可以低成本补上。
安全 / 性能
无新攻击面:没有任何路径来自模型(deadline.ts 与 retirement.ts 都从 planPath / CLI 导出的环境变量推导,与模块其余部分一致),无新增网络或 shell。第 3 轮起每个 --all-chunks 轮多一次 readTranscripts——对一个已在 Step 3D 与 Step 6 读过的目录做 O(transcripts) 解析,相对于数十分钟的一轮可忽略。按所示网格折算的收益看起来可信;流水线那一半更可靠,因为它不依赖 chunk 变冷。
|
@qwen-code /takeover |
Code Coverage Summary
CLI Package - Full Text ReportCore Package - Full Text ReportFor detailed HTML reports, please see the 'coverage-reports-22.x-ubuntu-latest' artifact from the main CI run. |
qwen-code-ci-bot
left a comment
qwen-code-ci-bot
left a comment
There was a problem hiding this comment.
LGTM, looks ready to ship — CI landed green after the review. ✅
|
🤝 Takeover engaged: the autofix loop now manages this PR — it will address new review feedback and resolve base conflicts until the label is removed or the round cap is reached. Remove the 中文说明🤝 已接管：autofix 循环现在管理此 PR —— 将持续处理新的评审反馈与 base 冲突，直到移除标签或达到轮次上限。移除 |
qwen-code-dev-bot
added
the
autofix/takeover
|
🤖 Addressed the latest review feedback (round 1/100). What changed, and what I pushed back on: · 已处理最新评审反馈（第 1/100 轮）。改动内容与我反驳保留之处如下： Address-review summary — PR #8498 (round after @wenshao's review)All feedback points from the review are addressed below. Both Criticals are Critical 1 — budget state survives into the CI in-job retry (FIXED)Both on-disk budget states —
The fence was chosen over clearing Tests: Critical 2 — one transcript can retire every chunk (FIXED)
Test: Suggestion 1 — staggered parities make CONVERGED unreachable (FIXED)Cold checks now land on ONE global parity — the even rounds — instead of each Suggestion 2 — pipelining suppresses one round of rediscovery for a later-rejected entry (ADDRESSED)The SKILL's merge bullet now names the trade explicitly: "an entry a verifier Suggestion 3 — the budget gate refuses single-chunk repair rebuilds (FIXED)
Suggestion 4 — header truncation self-check no longer describes the output (FIXED)The Smaller notes
Deferred
Failed checks noteAll checks listed as failed for this PR are Verification
中文说明处理审查意见总结 — PR #8498（@wenshao 评审后的一轮）评审中的每一条意见都在下面逐条处理。两个 Critical 均已修复，并以可复现的测试钉死；四条 Suggestion 全部实现；其他小点中两条已修、一条带证据拒绝、一条无需改动。一项测试覆盖缺口带理由推迟。 Critical 1 — 预算状态带入 CI 同 job 重试（已修复）两份磁盘预算状态 ——
选择围栏而非在 capture 时清空 测试： Critical 2 — 一份 transcript 可以让所有 chunk 退休（已修复）
测试： 建议 1 — 奇偶错位让 CONVERGED 永远不可达（已修复）冷查现在落在同一个全局奇偶上 —— 偶数轮 —— 而不再是每个 chunk 自己证书的奇偶。各自的锚永远无法重新对齐（一个 dry 于 2、3 轮的 chunk 与一个 dry 于 1、2 轮的 chunk 会在相反的轮次永远轮流冷查），于是在错位的大 PR 形状上 all-retired 的 建议 2 — 流水线化会让被否决条目压制一轮复查（已处理）SKILL 的合并 bullet 现在点名这个代价："an entry a verifier later rejects will have suppressed one round of rediscovery in its neighbourhood — the window is one round in one location, and the 5-round cap still bounds the loop。"PR 描述里"检测语义不变"的措辞不是本轮可编辑的文件；SKILL 文本（下一次运行真正会读的部分）已更正。 建议 3 — 预算闸拒绝单 chunk 的修复重建（已修复）
建议 4 — 表头截断自检已不描述实际输出（已修复）
其他小点
推迟项
关于失败检查本 PR 列出的失败检查全部是 验证
Base-conflict check · 基分支冲突检查: no conflict with main. · 与 main 无冲突。 Re-review when you have a moment. After round 100 this bot stops and leaves the PR for a human. · 有空请复审；第 100 轮后本 bot 停止并将 PR 交给人工。 🧠 Handled by Qwen Code · model/模型 |
Review —
|
|
🤖 Addressed the latest review feedback (round 2/100). What changed, and what I pushed back on: · 已处理最新评审反馈（第 2/100 轮）。改动内容与我反驳保留之处如下： Review feedback addressed — PR #8498Feedback source: @wenshao's issue-level review comment (2026-08-04). No inline 🔴 Critical — the anti-shortcut guard was inverted (retirement.ts)Fixed. The guard counted transcripts per record ( 🟡 Suggestions
🔵 Minor
Failed checks
Verification
中文说明已处理的审查反馈 — PR #8498反馈来源：@wenshao 的 issue 级审查评论（2026-08-04）。本轮没有行内评论，所有处置记录在此。未要求解决 base 冲突（ 🔴 Critical — 反捷径守卫方向反了（retirement.ts）已修复。 守卫统计的是每条 record 被几条 transcript 匹配（ 🟡 建议
🔵 小项
失败检查
验证
Base-conflict check · 基分支冲突检查: no conflict with main. · 与 main 无冲突。 Re-review when you have a moment. After round 100 this bot stops and leaves the PR for a human. · 有空请复审；第 100 轮后本 bot 停止并将 PR 交给人工。 🧠 Handled by Qwen Code · model/模型 |
ReviewReviewed the two commits this PR owns on top of #8468 ( Verified locally
What works well
Suggestion 1 —
|
|
🤖 AutoFix ran out of time before finishing (timeout (7200000ms)) (attempt 3/100) — it will retry on the next scan.
See the Qwen Autofix agent step logs for model/tool output. Run log: https://github.com/QwenLM/qwen-code/actions/runs/30897484008 🧠 Handled by Qwen Code · model/模型 |
|
@qwen-code /takeover stop |
qwen-code-dev-bot
removed
the
autofix/takeover
|
👋 Takeover released: the autofix loop will no longer engage this PR (an in-flight round, if any, completes its bounded work). Re-apply 中文说明👋 已释放：autofix 循环不再介入此 PR（在飞的一轮如有，将完成其有界工作）。重新打上 |
|
All four Suggestions and the Nice-to-haves are in, across S1 — the S2 — S3 — the classifier now reads the receipt's structure: the no-issues phrase (en variants + S4 — the cumulative list tags every not-yet-verified entry NTHs — transcripts are pre-filtered on the builder's role marker and each survivor's launch prompt is flattened once ( Two more rounds of internal review on the amended head surfaced one further hazard, fixed in
— via Claude Code (claude-fable-5) |
|
@qwen-code /takeover |
qwen-code-dev-bot
added
the
autofix/takeover
|
🤝 Takeover engaged: the autofix loop now manages this PR — it will address new review feedback and resolve base conflicts until the label is removed or the round cap is reached. Remove the 中文说明🤝 已接管：autofix 循环现在管理此 PR —— 将持续处理新的评审反馈与 base 冲突，直到移除标签或达到轮次上限。移除 |
|
@qwen-code /triage |
|
Sandboxed verification: ❌ not passed — findings reported (agent verdict) - workflow run Ran the PR in an isolated, token-free container: A/B against the base build, mock-free harness assertions, targeted gates. Advisory evidence for human reviewers — not a review, an approval, or a CI check. Scripted assertions: 167 passed · 1 failed · 168 total 中文 — 判定：❌ 不通过 · 报告了发现（agent 判定）沙箱验证在隔离、无凭证的容器中执行了该 PR 的代码（与 base 构建 A/B 对照、无 mock harness 断言、定向门禁）。仅作为评审证据，不构成评审、批准或 CI 检查。 脚本断言：167 通过 · 1 失败 · 168 总计 Verification reportPR 8498 verification —
|
| Cell | Scenario (round built) | base (control) | head | |
|---|---|---|---|---|
| C1/C2 | rounds 1-2, any history | 3 auditors, exit 0 | identical | PASS |
| C3 | all chunks dry ×2 → r3 | 3 built, exit 0 | exit 5, CONVERGED, 0 built, 0 records, 0 stamps | PASS |
| C4 | chunks 1,2 dry ×2, 3 yielded → r3 | 3 built | 1 built (chunk 3); note names 1,2, next cold check 4; records = [3] | PASS |
| C5 | same history → r4 (even) | 3 | 3, chunks 1,2 tagged (cold check) |
PASS |
| C6 | chunk 2 whiffed r2 → r3 | 3 | 2 ([2,3]); note names only chunk 1 | PASS |
| C7 | ONE transcript carrying both r2 records → r3 | 3 | 3 (attack certifies nothing) | PASS |
| C7b | two honest transcripts, one record → r3 | 3 | 1 ([3]) (relaunch merge kept) | PASS |
| C8 | transcripts env absent → r3 | 3 | 3 (fail open) | PASS |
| C9 | expired deadline, due round → r3 | 3, exit 0 (env ignored) | exit 4, BUDGET line, marker written, 0 records/stamps | PASS |
| C10 | expired deadline + converged → r3 | 3, exit 0 | exit 5 (not 4), no marker | PASS |
| C11a | --chunk 1 --round 3 of a stamped round, expired |
exit 0 | exit 0, record written (repair bypasses) | PASS |
| C11b | --chunk 2 --round 4 unstamped, expired |
exit 0 | exit 4 (unadmitted round is gated) | PASS |
| C12a/b | round 1, remaining 2100 / 800, reserve 600 | 0 / 0 | exit 0 (scaled 700) / exit 4 | PASS |
| C15 | plan recaptured after history (retry fence) → r3 | 3 | 3 (stale records fenced, full audit) | PASS |
| G-zh / G-short | zh receipt / 41-char backtick receipt | 3 | retired ([3]) | PASS |
| G-bare / G-notools / G-wrongfile | bare receipt / no tool calls / calls that never open the diff | 3 | hot ([1,3]) | PASS |
| G-yield | filed-finding echo (**File:** N/A vs real) |
3 | hot ([1,3]) | PASS |
| C14b | marker, coverage clean | 0 occurrences | 1 occurrence (disclosure renders) | PASS |
| C14c | marker + orchestrator-relayed copy | ≥1 | 1 (deduped, not doubled) | PASS |
| C14 | marker + reverse audit coverage gap |
0 | 0 — disclosure suppressed | FAIL (finding) |
The skipped-chunk rounds write no prompt records (C4: records = [3]), so the coverage roster owes nothing for retired chunks — the PR's check-coverage interplay claim holds by construction.
Findings
C14 — the budget-stop disclosure is silently dropped when a coverage subject collides (bounded; Suggestion-level)
composeReviewBody synthesizes the verdict-capping entry from the marker and pushes it into unreviewed. The pre-existing caller-vs-coverage dedupe then drops any entry that startsWith('<coverage subject> — '). The marker's entry is reverse audit — stopped before round N by the review time budget, and coverage emits subject reverse audit whenever a reverse-audit delivery is not ok (brief-unread / rewritten / not-launched — coverage.ts:1311). So on exactly the runs where the audit both mis-delivered and hit the budget gate, the body discloses the mis-delivery but never says the loop was truncated — the deterministic-disclosure property the PR's own commit message promises ("a run that drops the sentence still cannot approve past a truncated audit") loses its sentence.
Bounded: the suppression requires the colliding coverage entry to exist, and that entry itself caps the verdict (cappedBy: unreviewed-dimension), so an Approve past a truncated audit is still impossible — verified in both states (verdictStillCapped: true). The stderr BUDGET: line and the marker file also survive; only the body's wording is lost. The condition is not exotic in the gate's own terms: whiffed/mis-delivered auditors are precisely the runs whose extra rounds push the loop into the budget gate.
Reproduce (real CLI, both halves): node harness/ab.mjs <art-dir> — cell C14 vs C14b (witness 03-c14-disclosure-collision.png). Minimal shape: two dry rounds whose round-2 transcripts never open their brief, then a refused round 3 (QWEN_REVIEW_DEADLINE_EPOCH in the past), then qwen review compose-review --input {planPath, verdict, modelId} --comments []. Head prints the entry 0 times; with brief-opening transcripts (C14b) it prints 1.
Minimal suggested fix (measured in a scratch tree; not applied to the PR)
Remember the marker entry and exempt its exact string from the coverage-echo suppression in compose-review.ts:
let budgetStopEntry: string | null = null;
if (input.planPath) {
const stop = readBudgetStop(input.planPath);
if (stop !== null) {
budgetStopEntry = stop.entry;
if (!unreviewed.some((u) => u.includes(stop.entry))) unreviewed.push(stop.entry);
}
}
// …in the render loop, before echoesCoverage:
if (d === budgetStopEntry) { callerLeft.push(d); continue; }Measured (witness 04-fixcheck-patched-build.png): collision 0→1 occurrence; clean 1→1; relay+marker 1→1 (dedupe preserved); compose-review.test.ts 152/152 green both with and without the patch — the suite pins nothing along this axis, so the fix should ship with a fixture in the C14 collision shape (coverage gap reverse audit + marker).
Completeness reporting (not merge conditions): three unpinned guards
Mutation matrix over the scratch tree (witness 02-mutation-matrix.png; positive control M0 killed 8 tests, so the harness provably fails the suite): M1 injectivity, M2 non-empty convergence, M3 global parity, M5 round+reserve, M6 round-1 scaling, M8 record fence all killed by the PR's own tests with the intended expected-vs-actual failures. Three survivors, each classified:
- M4/M4b —
diffToolCalls > 0clause unpinned. The only fixture meant to pin it (retirement.test.ts:223) uses zero tool calls, which pinssuccessfulToolCalls > 0instead. The clause is live, not dead: harness cell G-wrongfile (substantive receipt, one successful call on a file that is not the diff) keeps the chunk hot on head. Coverage gap; the G-wrongfile shape is the fixture that would go red. - M7 — marker-vs-relayed dedupe unpinned (removing it stays 152/152). The behaviour is live: C14c renders exactly one occurrence with both channels. Coverage gap; a relay+marker fixture would pin it.
- M10 — the bare-subject vs
subject —distinction unpinned. This is the axis C14's collision rides on (the policy itself is pinned: M9, disabling the echo-dedupe, killed 3 pre-existing tests). So C14 is a new writer landing on the wrong side of a pinned, pre-existing merge policy — the author inherits a collision the policy's tests never name.
Not covered
- Per-commit attribution. Shallow graft (18 commits in metadata, 1 reachable); the aggregate effective diff is what was verified, including the stacked fix(review): stop the reverse-audit loop while there is still time to report #8468 gate.
- Change 2 (pipelining) has no behavioural oracle — it is orchestrator prose. Checked textually only: the SKILL.md diff keeps the two load-bearing orderings (last round's verification before Step 6;
— [unverified]tags structurally excluded from Step 6's confirmed-only read). The projected 95→55-minute saving is the author's arithmetic, not measured here. - Gates: only the affected workspace suite ran —
src/commands/reviewat head: 1748 passed, 4 skipped (1752) (the 4 skips are pre-existing; the PR's "1710" figure predates later merges). Repo-wide lint/typecheck/CI not re-run. - Workflow bash replay is uncalibrated: the reserve ladder (600 floor / quarter / 3600 cap, 7/7 assertions) and
bash -nran underset -euo pipefail, but no real emitted artifact exists to byte-match against (first round, noprevious-report.md). The 3600 cap mirroringDEFAULT_RESERVE_SECONDSis pinned bydeadline.test.ts:329. - Synthetic transcripts, not model behaviour. The A/B proves the scheduler's handling of every receipt shape; it does not reproduce real auditor degradation (shape, not cause).
- Base-arm type noise is environmental, not an arm difference: a from-scratch
tsc --buildof the base worktree reports declaration errors (rootnode_modules/ajvis v6; v8 lives inpackages/core/node_modules, absent in worktrees). Emitted JS was complete and smoke-verified; the same from-scratch recompile at head shows the same noise, and head's own build trusts its tsbuildinfo.
Methodology
Environment: node:22-bookworm container, npm ci + npm run build pre-run at the merge commit; base control rebuilt only packages/cli in tmp/base-tree (874e46d); mutations and the measured fix ran in tmp/mut-tree at the merge commit with vitest resolving from the root node_modules (core's package-local node_modules symlinked in after the ajv-v6 worktree artifact was diagnosed). Every cell spawns the real compiled CLI as a child process with QWEN_CODE_PROJECT_DIR/QWEN_CODE_SESSION_ID pointed at per-cell state dirs; transcripts are JSONL in the harness's own format, launch prompts copied byte-for-byte from the CLI's recorded prompts so deliveredVerbatim pairing is honest. Assertions: 156 (A/B) + 4 (fix check) + 8 (workflow replay) = 168, of which 1 failed (C14). Raw logs per cell in logs/, harnesses in harness/ (ab.mjs, fixcheck.mjs, mutations.sh, workflow-replay.sh, capture emitters), evidence in evidence/.
Evidence images
Harness scripts and raw logs are in the workflow run artifacts (7-day retention).
— Qwen Code · sandboxed verification
qwen-code-ci-bot
left a comment
qwen-code-ci-bot
left a comment
There was a problem hiding this comment.
LGTM, looks ready to ship. ✅
…e verification Rebuild of the retirement/pipelining feature as one commit on top of the merged reverse-audit budget gate (#8468). - retirement.ts: per-chunk scheduler over the CLI's own prompt records and the harness transcripts — a chunk whose two most recent audits are substantive dry receipts is cold-checked on the even rounds instead of audited on every one, and a cold check that yields returns it to every-round auditing. The certifying match is counted records per transcript (one launch matching several records certifies none), dry receipts are read structurally with the zh forms beside the English ones, and a filed finding requires the File+Severity pair so an echoed quotation cannot pin a chunk hot. Everything fails toward auditing. - agent-prompt --all-chunks: requireAuditableChunks, then the schedule (round >= 3, fail-open to all-due on any error), then CONVERGED exit 5 (nothing built, no stamp, no marker), then the budget gate (exit 4 + marker), then the build. The admission stamp keeps the #8468 ordering and lands only after the build succeeds: a cold-check-only round that builds still stamps, a converged round never does, and a build that throws leaves no stamp. - agent-prompt --chunk: a round holding an admission stamp is repaired without gates or scheduling; an unadmitted round answers to the same sequence as --all-chunks (convergence, then budget), and its first chunk build is the round's admission — stamped after the build. - prompt-record: optional sinceMs fence on readRecordedPrompts (history readers only; coverage's obligation reads stay unfenced), plus the flattenPrompt/deliveredVerbatim split so the scheduler flattens each launch once instead of once per (record, transcript) pair. - deadline: doc-comments rewritten for the pipelined cadence — the admission-to-admission measure no longer contains a verification pass, so the tail reserve is the terminal round's only cover (replacing the 'deliberate margin' overlap rationale), and the workflow's reserve cap is cross-referenced. runEpochMs fencing, the bilingual budget-stop marker and the stamp semantics are unchanged from #8468. - SKILL Step 5: builder-owned 3B scheduling, the CONVERGED exit-5 termination rule, verification launched alongside the next round's auditors, and the cumulative reported list with '— [unverified]' tagging (added at the admitting merge, cleared or removed after the verdict; anything still tagged is excluded from Step 6). Superseded pieces of the parallel branch were dropped in favour of the #8468 form now on main: the planMtimeMs-equality fence (runEpochMs stays), the budget-scaled round-1 estimate, stamping inside the admission helper, and the branch's variants of the budget-gate tests.
b2c181a to
0cf3170
Compare
|
Please do not rebase or force-push to an active PR as it invalidates existing review comments. Note for future reference, the bots always squash all changes into a single commit automatically as part of the integration. 中文请勿对活跃的 PR 执行 rebase 或 force-push，因为这会使已有的评审评论失效。另外，供日后参考：作为集成流程的一部分，机器人始终会自动将所有改动压缩（squash）为单个提交。 |
|
🤖 Reviewed the latest feedback — no changes needed. Why, point by point: · 已审阅最新反馈——无需改动。逐点说明原因如下： Autofix round: no actionNo code changes were made this round. Every actionable point in the new feedback is already present at the PR head ( Feedback triage
The author reply states these fixes landed in commits The remaining issue-level comment is the sandboxed-verification status notice, which carries no actionable feedback. There are no reviews, no inline comments, and no failed or still-red checks this round. Verification (evidence for the no-action call)
中文说明Autofix 本轮：无操作本轮未做任何代码改动。新反馈中所有可操作的点都已存在于 PR head（ 反馈分诊
作者回复称这些修复落在提交 其余 issue-level 评论是沙箱验证的状态通知，不含可操作反馈。本轮没有 review、没有 inline 评论、没有失败或持续失败的检查。 验证（无操作结论的证据）
Base-conflict check · 基分支冲突检查: no conflict with main. · 与 main 无冲突。 🧠 Handled by Qwen Code · model/模型 |
|
@qwen-code /triage |
|
Sandboxed verification: ❌ not passed — findings reported (agent verdict) - workflow run Ran the PR in an isolated, token-free container: A/B against the base build, mock-free harness assertions, targeted gates. Advisory evidence for human reviewers — not a review, an approval, or a CI check. Scripted assertions: 136 passed · 1 failed · 137 total 中文 — 判定：❌ 不通过 · 报告了发现（agent 判定）沙箱验证在隔离、无凭证的容器中执行了该 PR 的代码（与 base 构建 A/B 对照、无 mock harness 断言、定向门禁）。仅作为评审证据，不构成评审、批准或 CI 检查。 脚本断言：136 通过 · 1 失败 · 137 总计 Verification reportPR 8498 verification (follow-up round) —
|
| # | Round-1 finding | Severity | Status at new head 0cf3170 (re-measured, not diffed) |
|---|---|---|---|
| C14 | Budget-stop disclosure silently dropped when a coverage subject reverse audit collides |
Suggestion (bounded) | Fixed — in base, not by this PR. compose-review.ts is untouched by this diff; the fix landed in main between rounds (base d2c7cb1 already carries it): the marker entry is now synthesized as a structural, bilingual coverage entry, the BUDGET_STOP_PHRASE splice dedupes relayed copies, and budgetEntry is exempted from the prefix echo filter. Re-measured at the new head with the round-1 scenario through the real CLI: collision / clean / relayed cells all render the disclosure exactly once and the verdict stays capped (COMMENT, no LGTM) — 8/8 assertions, identical on both arms (witness 03-c14-collision-fixed-both-arms.png). The pre-fix arm (874e46d era) is not reconstructible under the depth-2 graft, so the broken state is cited from round 1's measurement; both current arms plus the new pinning tests are this round's evidence. |
| M4/M4b | diffToolCalls > 0 clause unpinned by tests |
Coverage gap | Stands. Re-measured: deleting the clause leaves retirement.test.ts + agent-prompt.test.ts green (both files, 357-test baseline). The clause is live, not dead code: A/B cell G-wrongfile shows a substantive receipt whose one successful call opens a different file keeps the chunk hot on head (built=[1,3]). Coverage gap; the G-wrongfile shape is the fixture that would go red. |
| M7 | Marker-vs-relayed dedupe unpinned | Coverage gap | Superseded — now pinned. Removing the BUDGET_STOP_PHRASE splice now kills compose-review.test.ts ("a budget-stop marker caps APPROVE at COMMENT with nothing relayed by the caller" — the reshaped-relay assertion fails). The test landed in main together with the C14 fix (witness 04-mutation-matrix.png). |
| M10 | Bare-subject vs subject — dedupe distinction unpinned |
Coverage gap | Stands. Re-measured: removing the `d === e.subject |
Central claim + A/B
Central claim (this PR): from round 3, agent-prompt --all-chunks becomes the loop's scheduler — a chunk whose two most recent audits were substantive dry receipts (receipt names what it examined AND the transcript shows the diff was opened) is retired to alternating even-round cold checks, returns to every-round auditing on any yield, and an all-retired round exits 5 (CONVERGED) with nothing built, recorded or stamped; every classification failure falls toward auditing. Secondary claims verified: convergence outranks the budget gate (exit 5, no marker); a stamped round's --chunk repair bypasses the expired gate while an unstamped --chunk round answers the same convergence-then-budget sequence; the retry fence (records/transcripts older than a re-captured plan read as absent).
Both arms drive the real compiled CLI (packages/cli/dist/index.js) over synthetic 3-chunk review states; history is established by running each arm's own builder for rounds 1-2 and delivering harness-shaped transcripts (launch prompt = the CLI's own recorded block with separators, tool calls and final text authored per scenario). Base = tmp/base-tree at d2c7cb1 with packages/cli rebuilt there (see Methodology for the control-cleanliness assertion). All cells re-ran identically under the evidence captures.
Table 1 — A/B cells (82 assertions, all passed; base control behaviors are asserted expectations, not failures). Witnesses: 01-ab-head-central-flips.png, 02-ab-base-controls.png.
| Cell | Scenario (round built) | base d2c7cb1 (control) |
head 0cf3170 |
|
|---|---|---|---|---|
| C1/C2 | rounds 1-2, any history | 3 auditors, exit 0 | identical | PASS |
| C3 | all chunks dry ×2 → r3 | 3 built, exit 0 | exit 5 CONVERGED, 0 built, 0 records, 0 stamps | PASS |
| C4 | chunks 1,2 dry ×2, 3 yielded → r3 | 3 built | 1 built (chunk 3); note names 1,2 (dry r1+r2, next cold check 4); records = [3]; stamped | PASS |
| C5 | same history → r4 (even) | 3, no tags | 3, chunks 1,2 tagged (cold check); tag never enters recorded prompts |
PASS |
| C6 | chunk 2 whiffed r2 → r3 | 3 | 2 built ([2,3]); note names only chunk 1 | PASS |
| C7 | ONE transcript carrying all three r2 records → r3 | 3 | 3 (concat attack certifies nothing) | PASS |
| C7b | whiff-then-dry relaunch for one record → r3 | 3 | 1 built ([3]) (relaunch merge kept) | PASS |
| C8 | transcripts env absent → r3 | 3 | 3 (fail open, exit 0) | PASS |
| C9 | expired deadline, due round → r3 | exit 4, BUDGET line, marker, nothing built | identical (gate reorganization is behavior-preserving) | PASS |
| C10 | expired deadline + all-retired → r3 | exit 4 + marker | exit 5 (not 4), no marker — convergence outranks the gate | PASS (flip) |
| C11a | --chunk 2 --round 2 of a stamped round, expired |
exit 4 (uniform gate refuses the repair) | exit 0, rebuilt (stamped-round repair bypass — new in this PR) | PASS (flip) |
| C11b | --chunk 2 --round 4 unstamped, expired |
exit 4 | exit 4 (unadmitted round is gated like --all-chunks) |
PASS |
| C12a/b | round 1, remaining 2600/2000, reserve 600 | exit 0 / exit 4 | identical (estimate 1800 + reserve arithmetic unchanged) | PASS |
| C15 | plan re-captured after dry ×2 history → r3 | 3 | 3 (stale records + transcripts fenced, full audit) | PASS |
| G-zh / G-short / G-echo-na | zh receipt / backtick receipt / **File:** N/A echo |
3 | retired ([3] built) | PASS |
| G-bare / G-notools / G-wrongfile / G-echo-real | bare receipt / zero tool calls / call opens another file / severity + real file echo | 3 | hot ([1,3] built) | PASS |
Skipped chunk-rounds write no prompt records (C4: records = [3] only), so the coverage roster owes nothing for retired chunks — the PR's check-coverage interplay claim holds by construction, as in round 1.
Corrections
None — nothing from the previous round or bot comments misdescribed the code in a way this round needs to correct. One metadata note: the PR body says "its own two commits"; the snapshot and the local tree both carry one (the two changes are one commit, 0cf3170). Not load-bearing — the effective diff contains both changes and was verified as the aggregate.
Findings
F1 — fullwidth-colon receipts never classify as dry: the separator class contradicts its own comment (Suggestion-level, bounded, fails toward auditing)
DRY_RECEIPT_RE's separator alternative is documented as admitting "a colon in either width" (retirement.ts, comment above the regex), but the compiled class is two ASCII colons — bytes 0x3a 0x3a, no U+FF1A:
'\\s*(?:[—–]+|[::]|--+|-+\\s)\\s*' + // [::] = 0x3a, 0x3a — fullwidth ： absent
Measured through the real compiled scheduler (harness/ladder.mjs; witness 05-fullwidth-colon-head-vs-patched.png): a receipt No issues found：re-walked the reconnect state machine… (and the zh-native shape 未发现问题：重新走查了…) classifies unknown, so the chunk never retires and is re-audited every round. All sibling separators pass: em/en dash, doubled em ——, ASCII colon, CRLF clause, and the zh phrase forms (没有发现新的问题, 无新问题) with dash separators.
Bounded: the failure direction is safe — unknown reads as not-dry, the chunk stays hot, no audit is ever skipped; this is a missed optimization, not a detection hole. But it lands exactly where the module's own history says it hurts: the comment records that the first cut's length floor failed on "a Chinese receipt of ANY length", and the switch to structural parsing was made so zh receipts could retire "on exactly the budgeted runs the optimization exists for". A zh auditor whose natural punctuation is ： gets no benefit from the change.
Reproduce: node tmp/pr8498-verify-20260804-153907/harness/ladder.mjs — the sibling fullwidth colon row FAILs on head (15/16); harness/ladder-patched.mjs (same harness against a scratch copy of the compiled module with the class fixed) is 16/16.
Minimal suggested fix (measured in a scratch copy; not applied to the PR)
'\\s*(?:[—–]+|[:：]|--+|-+\\s)\\s*' + // ASCII + fullwidth colonMeasured: hostile/benign matrix flips the fullwidth-colon case to retired and leaves every other variant byte-identical (16/16, witness 05-fullwidth-colon-head-vs-patched.png). The suite is green with and without the patch — nothing pins this axis; the fixture that would go red is a fullwidth-colon receipt case beside a Chinese receipt with a named territory is dry in retirement.test.ts, so the fix should ship with it.
Completeness reporting (carried over; not merge conditions): two unpinned guards, re-measured
- M4 —
diffToolCalls > 0clause unpinned (stands). Deleting it leavesretirement.test.ts+agent-prompt.test.tsgreen. Live, not dead: A/B G-wrongfile keeps the chunk hot when the only successful call opens a file that is not the diff. The only fixture aimed at the clause uses zero tool calls, which pinssuccessfulToolCalls > 0instead; the G-wrongfile shape is the fixture that would pin it. - M10 — bare-subject vs
subject —dedupe distinction unpinned (stands). Removing thed === e.subject ||branch leavescompose-review.test.tsgreen. The policy is live (bare caller subjects dedupe against coverage subjects, including the budget entry's), and this is the axis the round-1 C14 collision rode — worth a fixture when convenient.
Positive control: disabling retirement outright (const retired = false) turned 16+ tests red across both files with expected-vs-actual mismatches in the scheduling assertions, so the suite's green above is a measurement, not a vacuous pass (witness 04-mutation-matrix.png). Killed as before: cold-check parity inversion, empty-chunk convergence guard; newly killed: the marker-phrase splice (M7 row above).
Not covered
- Change 2 (verification pipelining) has no behavioural oracle — it is orchestrator prose in SKILL.md. Checked textually only: the two load-bearing orderings survive the rewrite (the last round's verification completes before Step 6; entries still tagged
— [unverified]are structurally excluded from Step 6's confirmed-only read; a rejected entry leaves the list at the next merge). The projected 95→55-minute loop saving is the author's arithmetic, not measured here. - Gates: only the affected workspace suite ran —
src/commands/reviewat head: 1755 passed | 4 skipped (1759, 54 files). The 4 skips are pre-existing (round 1 saw the same count family); the PR's "1710" figure predates later merges. Repo-wide lint/format/typecheck not re-run (PR claims clean). - Workflow replay not re-run: round 1's reserve-ladder replay was uncalibrated (no real emitted artifact); this PR's diff touches no workflow file (the
deadline.tsdelta is comment-only, documenting the workflow's reserve cap), so there was no new surface to replay. - Pre-fix compose arm not reconstructible: the depth-2 graft cannot reach the 874e46d-era code where C14 was measured broken; the fixed-state re-measurement (both arms, plus the pinning tests now in the suite) is this round's evidence, and the broken state is cited from round 1.
- Synthetic transcripts, not model behaviour. The A/B proves the scheduler's handling of every receipt shape; it does not reproduce real auditor degradation (shape, not cause).
- Base-arm build noise is environmental, not an arm difference: the base worktree's from-scratch
tsc --buildleaves one residual declaration error (@lydell/node-ptyTS7016, exports-vs-types resolution) after symlinking the package-localnode_modules; the emitted JS is complete and smoke-tested, and round 1 established the identical noise on a from-scratch head recompile. - No injection attempts observed in the PR title/body/commit message; they were treated as untrusted claims and tested, not followed.
Methodology
Environment: node:22-bookworm container, npm ci + npm run build pre-run at the merge commit; base control rebuilt only packages/cli in tmp/base-tree (d2c7cb1) with the head tree's package-local node_modules symlinked in (identical dependency trees — the PR touches no package.json/lockfile) and scripts/generate-git-commit-info.js re-run there. Control cleanliness asserted: from inside the base tree, @qwen-code/qwen-code-core resolves via the root node_modules symlink with realpath /__w/qwen-code/qwen-code/packages/core (the head tree), and git diff HEAD^1..HEAD -- packages/core is SKILL.md only — no JS — so the base runtime executes unchanged core code. Mutations ran in tmp/mut-tree at the merge commit (exact single-occurrence string replacements, restored after each run). Every A/B / C14 cell spawns the real compiled CLI as a child process with per-cell state dirs (QWEN_CODE_PROJECT_DIR/QWEN_CODE_SESSION_ID); transcripts are JSONL in the harness's own format with launch prompts copied byte-for-byte from the CLI's recorded blocks so deliveredVerbatim pairing is honest; the ladder imports the compiled dist scheduler module directly. Counts: 82 (A/B) + 16 (C14) + 16 (ladder, head) + 16 (ladder, patched fix check) + 7 (mutation-matrix expectations incl. baseline-green) = 137, of which 1 failed (F1). Raw per-cell logs in logs/ (ab-*.json, c14-*.json, ladder.txt, mutations*.txt, suite-head.log, base-build.log), harnesses in harness/ (ab.mjs, c14.mjs, ladder.mjs, ladder-patched.mjs, mutations.sh, mutations-detail.sh), evidence images in evidence/.
Evidence images
Harness scripts and raw logs are in the workflow run artifacts (7-day retention).
— Qwen Code · sandboxed verification
|
🤖 Reviewed the latest feedback — no changes needed. Why, point by point: · 已审阅最新反馈——无需改动。逐点说明原因如下： Autofix review round — no action requiredTriage result for this round: no actionable feedback, so no code changes were made.
The branch stays at commit 中文说明Autofix 审查轮次 —— 无需处理本轮分诊结果：没有需要处理的反馈，因此未做任何代码改动。
分支保持在提交 Base-conflict check · 基分支冲突检查: no conflict with main. · 与 main 无冲突。 🧠 Handled by Qwen Code · model/模型 |
yiliang114
left a comment
yiliang114
left a comment
There was a problem hiding this comment.
LGTM, no blockers. Retirement engine verified correct and fail-open-toward-audit throughout; full review suite passes (1750/0) at head; CONVERGED/exit-5 semantics and the bypass plug are sound. Non-blocking: (1) P2 deadline.ts:191 — expectedRoundSeconds under-prices a round admitted after a shrunk (retired) round, same under-reserve direction #8368 called unsafe; price from max(observed spans) or note the interaction. (2) P2 SKILL.md:623 — the '[unverified]' exclusion is prose-enforced, not code-enforced (nothing parses the tag); Suggestions lack the criticals-unverified backstop, so add a compose-review check or soften 'structurally excluded'.
doudouOUC
left a comment
doudouOUC
left a comment
There was a problem hiding this comment.
Reviewed. Suggestions are inline.
中文说明
已审查。 建议见行内评论。
— claude-opus-4-6 via Qwen Code /review (v0.21.5)
|
|
||
| it('a return that never opened the diff is not dry, however substantive it sounds', () => { | ||
| transcript(record(1, 13, 'chunk 13 round 1 territory walk'), DRY); | ||
| transcript(record(2, 13, 'chunk 13 round 2 territory walk'), DRY, 0); |
There was a problem hiding this comment.
[Suggestion] No test distinguishes diffToolCalls > 0 from successfulToolCalls > 0 — the classifier's two independent guards are exercised only in lockstep. — Failure scenario: the transcript() helper always targets the diff path, so diffToolCalls === successfulToolCalls in every test. If rec.diffToolCalls > 0 were mutated to true, an auditor that calls read_file on non-diff files (e.g. its own brief) but never opens the diff would be classified dry instead of unknown, potentially retiring a chunk on the strength of a review that never read its territory.
Add a test case with a transcript whose successful tool calls target a non-diff file path and verify the schedule still treats the chunk as not-dry.
中文说明
没有测试能区分 diffToolCalls > 0 与 successfulToolCalls > 0 —— 分类器的两个独立守卫始终被联动测试。失败场景：transcript() helper 始终以 diff 路径为目标，因此每条测试 transcript 中 diffToolCalls === successfulToolCalls。如果将 rec.diffToolCalls > 0 突变为 true，一个调用了 read_file 但从未打开 diff 的 auditor 会被分类为 dry 而非 unknown，可能在从未读取其领地的审查基础上退休一个 chunk。
建议添加一个测试用例：transcript 中有成功的 tool call 但目标为非 diff 文件路径，验证调度器仍将其视为 not-dry。
— claude-opus-4-6 via Qwen Code /review (v0.21.5)
|
Qwen Code review completed for this PR. Verdict: Comment (no blockers) Summary: 1 Suggestion — missing test coverage for Build: all 18 workspaces passed. CLI test suite hit the 300s budget; core had 1 pre-existing failure unrelated to this PR. |
… tests Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
|
🤖 Addressed the latest review feedback (round 1/100). What changed, and what I pushed back on: · 已处理最新评审反馈（第 1/100 轮）。改动内容与我反驳保留之处如下： Review feedback addressed — PR #8498Feedback points[rc:3714551707] [Suggestion]
|
qwen-code-ci-bot
left a comment
qwen-code-ci-bot
left a comment
There was a problem hiding this comment.
Not reviewed: build-and-test — Integration Tests (CLI, No Sandbox) was skipped in CI and its suite did not run locally.
Test Plan (not a blocker): 1710 passed — this review observed 19117 passed.
中文说明
未审查：build-and-test — Integration Tests (CLI, No Sandbox) was skipped in CI and its suite did not run locally。
Test Plan（非阻断）：1710 passed — this review observed 19117 passed。
— qwen3.8-max via Qwen Code /review (v0.21.5)
| '|没有(?:发现)?(?:新的?)?问题)' + | ||
| '\\s*(?:[—–]+|[::]|--+|-+\\s)\\s*' + |
There was a problem hiding this comment.
[Critical] The separator class [::] contains two ASCII colons (U+003A ×2) — the full-width colon ： (U+FF1A) is missing, although the doc comment above claims "a colon in either width". — Failure scenario: a substantive Chinese dry receipt separated by the standard full-width colon — e.g. 未发现问题：重新走查了重连状态机与两个已改导出的全部调用点 — fails DRY_RECEIPT_RE, so classifyReturn returns unknown instead of dry: the chunk never retires and re-audits every round — the exact silent-cost mode this module's header comment says the rewrite exists to eliminate. Probed on unmodified code: full-width-colon receipt → due:[13]; byte-identical receipt with an ASCII : → retired; the fix below flips the zh arm to retired with all 22 tests still green. Audit-safe direction (cost, not skipped audits), but the classifier rejects an input its own contract admits, and no existing test covers the full-width colon.
| '|没有(?:发现)?(?:新的?)?问题)' + | |
| '\\s*(?:[—–]+|[::]|--+|-+\\s)\\s*' + | |
| '|没有(?:发现)?(?:新的?)?问题)' + | |
| '\\s*(?:[—–]+|[:：]|--+|-+\\s)\\s*' + |
中文说明
分隔符字符类 [::] 包含两个 ASCII 冒号（U+003A ×2）——缺少全角冒号 ：（U+FF1A），尽管上方文档注释声称"任一宽度的冒号"。失败场景：以标准全角冒号分隔的中文实质干回执（如 未发现问题：重新走查了重连状态机与两个已改导出的全部调用点）无法匹配 DRY_RECEIPT_RE，classifyReturn 返回 unknown 而非 dry：该 chunk 永不退休、每轮重审——正是本模块头注释声称此次重写要消除的静默成本模式。已对未修改代码做探针验证：全角冒号回执 → due:[13]；字节相同但用 ASCII : 的回执 → 退休；下方修复使中文分支转为退休，且全部 22 个测试仍然通过。失败方向是审计安全的（成本而非漏审），但分类器拒绝了自身契约本应接受的输入，且现有测试均未覆盖全角冒号。
— qwen3.8-max via Qwen Code /review (v0.21.5)
| // Move the fence past the transcripts: a new capture rewrote the plan. | ||
| const now = new Date(Date.now() + 60_000); | ||
| utimesSync(plan, now, now); |
There was a problem hiding this comment.
[Suggestion] This test does not exercise the transcript fence: advancing the plan mtime to now + 60s fences out the records too (readRecordedPrompts uses the same fence), so due:[13] passes with zero records regardless of transcripts. — Failure scenario: probe — deleting the since argument from the scheduler's readTranscripts call leaves all 22 tests green, so the fence has no non-vacuous coverage; the exact collision the fence exists for (a second /review in the same session whose prior run's dry transcripts verbatim-match the fresh run's records) could then retire chunks never audited this run. The fix is probe-verified: the fence time must land between the transcript writes and the re-records (plan mtime = now), with identical record bodies — a literal "re-record at now+60s" stays vacuous because those files are still older than the future-dated plan.
| // Move the fence past the transcripts: a new capture rewrote the plan. | |
| const now = new Date(Date.now() + 60_000); | |
| utimesSync(plan, now, now); | |
| // Move the fence past the transcripts, then re-record with identical | |
| // bodies: the records stay fresh while only the transcripts age out. | |
| const now = new Date(); | |
| utimesSync(plan, now, now); | |
| record(1, 13, 'chunk 13 round 1 territory walk'); | |
| record(2, 13, 'chunk 13 round 2 territory walk'); |
中文说明
该测试并未真正检验 transcript 围栏：把 plan 的 mtime 推到 now + 60s 同时也会把 records 隔在围栏外（readRecordedPrompts 使用同一围栏），因此无论 transcripts 如何，due:[13] 都会以零 records 通过。失败场景：探针表明——删除 scheduler 中 readTranscripts 调用的 since 参数后，全部 22 个测试仍然通过，即该围栏没有非空洞的覆盖；围栏本要防御的确切冲突（同一 session 中第二次 /review，其前次运行的干 transcripts 与本次运行的 records 逐字匹配）届时可能让本次运行从未审计过的 chunk 退休。修复已经探针验证：围栏时刻必须落在 transcript 写入与重新 record 之间（plan mtime = now），且 record 内容完全相同——字面上的"在 now+60s 重新 record"仍是空洞的，因为那些文件仍然早于未来时刻的 plan。
— qwen3.8-max via Qwen Code /review (v0.21.5)
| expect(readRoundStamps(plan).filter((s) => s.round === 3)).toHaveLength(1); | ||
| }); | ||
|
|
||
| it('gates a --chunk build of a round never admitted — no stamp, no exemption', () => { |
There was a problem hiding this comment.
[Suggestion] The --chunk gate's convergence-before-budget ordering has no test under deadline pressure: every expired-deadline + --chunk test has either a stamped round or no convergence history, and every converged + --chunk test has no deadline set (with DEADLINE_ENV unset, reverseAuditBudgetExhausted returns null, so even a budget-first reordering passes). — Failure scenario: a future edit moving admitReverseAuditRound above the schedule.converged check in the --chunk gate (agent-prompt.ts:1957-1961) exits 4 and writes a budget-stop marker for a review that had actually converged — compose-review then caps the verdict with a false "stopped before round N" truncation disclosure, and every existing test still passes. The current order is correct; this is the missing pin. Suggested sibling test, driving the handler with chunk: 13:
answerRound(1, { 13: DRY, 14: DRY, 15: DRY });
answerRound(2, { 13: DRY, 14: DRY, 15: DRY });
process.env[DEADLINE_ENV] = String(Math.floor(Date.now() / 1000) + 60);
call('reverse-audit', { chunk: 13, round: 3 }, plan);
expect(process.exitCode).toBe(5); // CONVERGED, not BUDGET
expect(readBudgetStop(plan)).toBeNull();中文说明
--chunk 门"先收敛后预算"的顺序没有在期限压力下被测试钉住：所有"期限已过 + --chunk"的测试要么持有 stamp、要么没有收敛历史，而所有"已收敛 + --chunk"的测试都没有设置期限（DEADLINE_ENV 未设置时 reverseAuditBudgetExhausted 返回 null，即使把顺序改为预算优先也能通过）。失败场景：未来某次修改把 --chunk 门中的 admitReverseAuditRound 提到 schedule.converged 检查之上（agent-prompt.ts:1957-1961），会 exit 4 并为实际已收敛的评审写下预算停止标记——compose-review 随后用虚假的 "stopped before round N" 截断披露为裁决设上限，而所有现有测试仍然通过。当前顺序是正确的；缺的是这个钉住测试。建议增加一个以 chunk: 13 驱动 handler 的同族测试。
— qwen3.8-max via Qwen Code /review (v0.21.5)
| const receipt = DRY_RECEIPT_RE.exec(text); | ||
| if ( | ||
| rec.successfulToolCalls > 0 && | ||
| rec.diffToolCalls > 0 && |
There was a problem hiding this comment.
[Suggestion] The dry evidence bar is range-blind: diffToolCalls > 0 accepts any successful read of the diff file wherever in it, never comparing the lines actually read against the chunk's range. — Failure scenario: probe on unmodified code — records baking the territory as offset=1000, limit=200, transcripts whose only diff read was offset=0, limit=50 plus a substantive receipt → the chunk retired and the loop converged though no auditor ever opened its territory; requiring a diffReads entry to overlap the baked offset/limit keeps it due while honest in-territory reads still retire. Nothing else catches it: roster.ts has no reverse-audit keys, and verificationGaps's floor is one agent opening its brief. Both fix halves already exist — AgentRecord.diffReads records the lines actually read, the matched record's prompt bakes the chunk's exact read, and coverage.ts has the expectedRange + containment machinery to mirror. Failure direction: skipped audits, contradicting the module's "never a skipped one" invariant.
中文说明
dry 证据标准对范围视而不见：diffToolCalls > 0 接受对 diff 文件任意位置的任何一次成功读取，从不把实际读到的行与该 chunk 的范围相比较。失败场景：对未修改代码的探针——records 把领地烘焙为 offset=1000, limit=200，而 transcripts 中唯一的 diff 读取是 offset=0, limit=50 加一份实质回执 → chunk 退休、循环收敛，尽管没有任何 auditor 打开过它的领地；若要求 diffReads 中至少一条与烘焙的 offset/limit 重叠，则它保持 due，而领地内的诚实读取仍然退休。没有其他环节能捕获：roster.ts 没有 reverse-audit 键，verificationGaps 的下限只是一个 agent 打开过 brief。修复的两半已经存在——AgentRecord.diffReads 记录了实际读到的行，匹配到的 record 提示词烘焙了该 chunk 的精确读取，coverage.ts 已有可参照的 expectedRange + 包含判断机制。失败方向：漏审，与模块"绝不漏审"的不变量相悖。
— qwen3.8-max via Qwen Code /review (v0.21.5)
| function substantiveClause(clause: string): boolean { | ||
| const c = clause.replace(/\s+/g, ' ').trim(); | ||
| if (c.length === 0) return false; |
There was a problem hiding this comment.
[Suggestion] substantiveClause is satisfiable by echoing the brief's own model receipt verbatim — the exact sentence every reverse auditor is handed in agent-briefs.ts ("No issues found — re-walked the reconnect state machine and the two changed exports' call sites; every gap I checked was already in the list") has no backticks, paths, or CJK, and clears the ≥20-char floor on length alone. — Failure scenario: probe — a transcript with an honest in-territory read whose final text is that sentence copied verbatim retires the chunk and converges the loop; rejecting that fixed sentence flips it to due. Combined with the range-blind diff bar above, a parrot whiff with one anywhere-read retires territory; the module's own measurements say agents parrot what they are handed. Fix note: the sentence is fixed text inside a template literal, not an exported constant — extract/export it first, then reject a clause that equals or verbatim-contains it.
中文说明
substantiveClause 可以被逐字回显 brief 自带的模型回执所满足——每个 reverse auditor 在 agent-briefs.ts 中都会拿到的那句示例（"No issues found — re-walked the reconnect state machine and the two changed exports' call sites; every gap I checked was already in the list"）不含反引号、路径或中日韩字符，仅凭长度就能越过 ≥20 字符的下限。失败场景：探针——一份在领地内诚实读取、最终文本为逐字复制的该句话的 transcript 会使 chunk 退休并让循环收敛；拒绝该固定句子后结果翻转为 due。与上方的范围视而不见的 diff 标准叠加，一只"鹦鹉学舌"式 whiff 加任意位置的一次读取即可让领地退休；本模块自己的测量表明 agent 会照抄被告知的内容。修复提示：该句子是模板字面量内的固定文本而非导出常量——先将其提取/导出，然后拒绝等于或逐字包含它的从句。
— qwen3.8-max via Qwen Code /review (v0.21.5)
| if (c.includes('`')) return true; | ||
| if (/\w[\w.-]+\/[\w.$/-]*\w/.test(c)) return true; |
There was a problem hiding this comment.
[Suggestion] substantiveClause's named-object shortcuts are satisfiable by prose artifacts: any stray backtick, and the path regex matching the conjunction "and/or". — Failure scenario: probe on unmodified code — No new issues found — all good. plus one trailing backtick (the exact receipt the 'clause names nothing' test pins as insufficient), and No issues found — and/or cases. (13 chars, under the 20-char floor) both retire chunk 13 and converge the loop at round 3; the artifact-free clause stays due. The fix flip — count only enclosed backtick spans (e.g. /[^]+/), and require the path heuristic to look like a path (a second slash or a dotted extension) — keeps all 22 tests green. Misjudgement here reads dry, contradicting both the doc comment's claim that it reads unknown` and the header invariant "never a skipped one".
中文说明
substantiveClause 的命名对象捷径可被行文伪影满足：任意一个游离反引号，以及路径正则匹配到连词 "and/or"。失败场景：对未修改代码的探针——No new issues found — all good. 加一个尾随反引号（恰是"从句未命名任何对象"测试钉住为不充分的那份回执），以及 No issues found — and/or cases.（13 字符，低于 20 字符下限）都使 chunk 13 退休并让循环在第 3 轮收敛；无伪影的从句保持 due。修复翻转——只计成对反引号包围的片段（如 /[^]+/），并要求路径启发式看起来像路径（有第二个斜杠或带点扩展名）——全部 22 个测试仍然通过。此处的误判读作 dry，与文档注释声称的读作 unknown` 以及头部"绝不漏审"的不变量均相悖。
— qwen3.8-max via Qwen Code /review (v0.21.5)
| if (SEVERITY_LINE_RE.test(text)) { | ||
| for (const m of text.matchAll(FILE_LINE_RE)) { | ||
| const file = (m[1] ?? '').trim(); |
There was a problem hiding this comment.
[Suggestion] Quoting a complete prior finding — **File:** plus **Severity:**, the exact shape every cumulative-list entry has — classifies an honest, evidence-backed dry return as yielded, so such a chunk can never retire; the SEVERITY_LINE_RE comment claims the pair "distinguishes a report from a quotation", but the cumulative list folded into the launch prompt consists of full blocks. — Failure scenario: probe — a return quoting a full cumulative-list entry to justify "already covered, not re-reporting" and ending with the substantive receipt → due:[13], no retirement ever; the same quotation minus the **Severity:** line retires and converges — the pair is the flip; a genuinely new finding still classifies yielded. Cost direction (over-audit), but the trigger is structurally far broader than the documented echo trade. Fix (probe-verified, all 22 tests green): before counting a File+Severity block as filed, skip any whose File line appears verbatim in rec.launchPrompt — the folded cumulative list is on the record.
中文说明
引用一条完整的先前发现——**File:** 加 **Severity:**，恰是累计清单中每个条目的形状——会把一份诚实且有证据支持的干返回分类为 yielded，于是这样的 chunk 永远无法退休；SEVERITY_LINE_RE 的注释声称该组合能"区分报告与引用"，但折叠进启动提示词的累计清单本身就是完整块。失败场景：探针——一份为说明"已覆盖、不再重复报告"而引用完整累计条目、并以实质回执结尾的返回 → due:[13]，永不退休；同一引用去掉 **Severity:** 行后退休并收敛——该组合即翻转开关；真正的新发现仍分类为 yielded。成本方向（过度审计），但其触发条件在结构上远比文档所述回显权衡宽广。修复（已探针验证，全部 22 个测试通过）：在把 File+Severity 块计为已提交之前，跳过 File 行逐字出现于 rec.launchPrompt 的块——折叠的累计清单就在 record 上。
— qwen3.8-max via Qwen Code /review (v0.21.5)
| - **On the 3B path the builder is also the convergence ledger**: when every chunk holds two consecutive substantive dry audits and none is due a cold check, `--all-chunks` builds nothing, prints a `CONVERGED` explanation to stderr and exits **5**. Stop the loop and proceed to Step 6 — this is a **clean** convergence, not a gap: no `unreviewedDimensions` entry is owed, and the per-chunk certificates are collectively the same evidence the global two-dry-rounds rule used to demand. | ||
| - Stop after **5 rounds** regardless (hard cap), and say so in the output rather than implying convergence. | ||
| - New findings from each round are merged into the cumulative list **before** the next round begins, so each round sees an updated baseline. | ||
| - Findings **reported** by each round are merged into the cumulative list **before** the next round begins, so each round sees an updated baseline. Verification status does not gate the merge — the list exists so auditors do not re-report what is already filed, and an unverified entry serves that purpose exactly as well as a confirmed one. The trade, named: an entry a verifier later rejects will have suppressed one round of rediscovery in its neighbourhood — the window is one round in one location, and the 5-round cap still bounds the loop. The tag is what keeps this mechanical rather than remembered: an entry enters the list tagged `— [unverified]`; the merge after its Step 4 verdict removes the tag (confirmed) or the entry (rejected). Step 6's confirmed-only read then has something to key on — anything still tagged is structurally excluded — instead of a memory of which round each entry arrived in. The tag rides inside the findings file, which is folded into the launch prompt and hashed into the record key, so no delivery check changes. |
There was a problem hiding this comment.
[Suggestion] Tag removal is scheduled only as a merge side effect, and merges are keyed to "findings reported by each round" — every loop-termination mode can arrive with the final rounds dry, so no merge pass runs after the last verdict lands. — Failure scenario: round k reports findings F_k (admitted tagged); its verifiers ride alongside round k+1's auditors and their confirmed verdicts land during k+1; rounds k+1 and k+2 report nothing, so the prose schedules no merge pass; the two-dry-rounds stop (or CONVERGED, or a budget stop) ends the loop with F_k still tagged — and the surviving-tag sentence seven lines down demotes it to terminal-only under a false "verifier failed" diagnosis: a finding the run's own verifier confirmed never reaches the PR body, and its only disclosure dies with the run. Fix: schedule the pass unconditionally — a merge pass runs before every round build and before Step 6 whether or not the last round reported findings, applying every landed Step 4 verdict (untag confirmed, remove rejected) — and reword the surviving-tag sentence: a surviving tag means the verdict was never applied; apply it from the verifier's return if one exists, and treat the entry as low-confidence terminal-only only when no verdict exists at all.
中文说明
去标签只作为合并的副作用被调度，而合并以"每轮报告的发现"为键——循环的每种终止方式都可能在末几轮干净的情况下到来，于是最后一个裁决落地之后再也没有合并发生。失败场景：第 k 轮报告发现 F_k（带标签进入清单）；它的 verifier 与第 k+1 轮的 auditor 同行，其确认裁决在 k+1 期间落地；第 k+1 与 k+2 轮都没有报告任何内容，于是按文案不会再有合并；两干净轮停止（或 CONVERGED，或预算停止）使循环在 F_k 仍带标签时结束——七行之后的幸存标签句又以虚假的"verifier 失败"诊断把它降为仅终端显示：一条已被本次运行自己的 verifier 确认的发现永远无法进入 PR 正文，其唯一披露随运行消亡。修复：无条件调度该合并——无论上一轮是否报告了发现，每次轮次构建之前与 Step 6 之前都运行一次合并，应用所有已落地的 Step 4 裁决（确认者去标签、否决者移除）——并改写幸存标签句：幸存标签意味着裁决从未被应用；若存在 verifier 的返回则从中应用，只有在根本不存在裁决时才把条目当作低置信度"仅终端显示"。
— qwen3.8-max via Qwen Code /review (v0.21.5)
| it('transcripts unavailable: full fan-out, never fewer', () => { | ||
| answerRound(1, { 13: DRY, 14: DRY, 15: DRY }); |
There was a problem hiding this comment.
[Suggestion] No test anywhere schedules a round ≥ 3 containing a chunk with NO audit history, so the lastTwo.length === 2 guard in retirement.ts that keeps history-less chunks hot is unpinned. — Failure scenario: probe on the pristine PR — chunk 13 twice dry plus a history-less chunk 16 → due:[16], correctly hot; reachable in production whenever a re-capture adds a chunk mid-loop or records for one chunk are lost while the loop continues. Verifier note: the named bare-.every regression actually crashes on the dryRounds read and degrades fail-open through the callers' try/catch (the CONVERGED-with-zero-audits shape needs a regression variant that also guards that read) — the core claim stands: a load-bearing guard with no pinning test.
// e.g. in retirement.test.ts — a history-less chunk must stay due
dryTwice([13]);
const r3 = schedule(3, [13, 16]); // 16 has no records
expect(r3.due).toEqual([16]);
expect(r3.skipped.map((s) => s.chunkId)).toEqual([13]);
expect(r3.converged).toBe(false);中文说明
任何测试都没有调度过"包含零审计历史 chunk"的第 3 轮及以上轮次，因此 retirement.ts 中让无历史 chunk 保持热区的 lastTwo.length === 2 守卫没有被钉住。失败场景：对原始 PR 的探针——chunk 13 两连干加上无历史的 chunk 16 → due:[16]，正确地保持热区；生产中可达：循环中途重新捕获新增了 chunk，或某个 chunk 的 records 在循环继续时丢失。验证者备注：所点名的裸 .every 回归实际会在读取 dryRounds 时崩溃，并经调用方的 try/catch 以失败开放方式降级（零审计即 CONVERGED 的形态还需要一个同时守卫该读取的回归变体）——核心主张成立：一个承重守卫没有钉住测试。
— qwen3.8-max via Qwen Code /review (v0.21.5)
| expect(r3.converged).toBe(false); | ||
| }); | ||
|
|
||
| it('several honest transcripts for ONE record all certify it — the relaunch merge', () => { |
There was a problem hiding this comment.
[Suggestion] The merge's multi-RECORD side is pinned nowhere: no test gives one (chunk, round) two live records with differing certified outcomes, so the history append implementing "any yield proves the territory hot" across same-round records is unguarded. — Failure scenario: reachable via this PR's own repair flow — a --chunk repair re-records the same (chunk, round) under a new findings digest (agent-prompt.ts:2004); retirement.ts's history comment names exactly this shape. Probe: original transcript YIELD + rebuild transcript DRY → due:[13] in both digest arrangements on current code; under an overwrite regression (byRound.set(rec.round, outcomesByRecord[i])) the entire existing suite stays green while the probe flips to due:[], skipped:[13], converged:true whenever the dry record iterates last — a yielded territory retires, decided by filesystem record order; CONVERGED can follow. Write the pair once per digest arrangement so an order-dependent overwrite cannot pass both.
it('two live records for ONE (chunk, round): any yield keeps it hot', () => {
transcript(record(1, 13, 'chunk 13 round 1 territory walk'), DRY);
transcript(record(2, 13, 'chunk 13 round 2 territory walk'), YIELD);
const rebuild = record(2, 13, 'chunk 13 round 2 territory walk (corrected rules)', 'rebuild01');
transcript(rebuild, DRY);
expect(schedule(3, [13]).due).toEqual([13]);
});中文说明
合并的多 RECORD 一侧没有任何钉住：没有测试为同一 (chunk, round) 提供两条认证结果相异的活 records，因此实现"任一 yield 即证明领地仍热"的跨同轮 records 历史追加无人守卫。失败场景：经由本 PR 自己的修复流程可达——一次 --chunk 修复会以新的 findings 摘要重新记录同一 (chunk, round)（agent-prompt.ts:2004）；retirement.ts 的历史注释点名的正是这一形状。探针：原始 transcript YIELD + 重建 transcript DRY → 当前代码下两种摘要排列均为 due:[13]；在覆盖式回归（byRound.set(rec.round, outcomesByRecord[i])）下整个现有套件仍然通过，而探针在干 record 最后迭代时翻转为 due:[], skipped:[13], converged:true——一个 yielded 领地退休，由文件系统 record 顺序决定；其后可以 CONVERGED。请按每种摘要排列各写一次该组合，使依赖顺序的覆盖无法同时通过两者。
— qwen3.8-max via Qwen Code /review (v0.21.5)
…ngs (#8498) Review-round fixes for the per-chunk retirement feature: - retirement.ts: admit the full-width colon (U+FF1A) in zh dry receipts, which the old [::] class silently refused; refuse the brief's own example receipt and prose artifacts (a stray backtick, the conjunction "and/or") in the substance check; require the transcript's diff reads to overlap the chunk's baked territory before a dry receipt counts; stop counting a quoted cumulative-list entry — a File+Severity block whose file line appears verbatim in the agent's own launch prompt — as a filed finding. Every change fails toward auditing. - agent-briefs.ts: export the reverse-audit example receipt (REVERSE_AUDIT_EXAMPLE_RECEIPT) and interpolate it in the brief, so the brief and the classifier's parrot refusal cannot drift. - Tests: rewrite the transcript-fence test, which a future-dated plan made vacuous, and add probe-verified pins for every fix plus the previously unpinned guards — converged-before-budget under deadline on the --chunk path, the --chunk transcripts-unavailable degrade, the stamp-keyed repair exemption (records without a stamp stay refused), budget-stop marker absence on the admission side, uncertified cold-check recovery, the history-less chunk guard, and the same-round multi-record merge in both digest orders. - SKILL.md: schedule the Step 5 findings merge unconditionally — verdicts land on dry rounds too — and cap a would-be Approve at COMMENT when a reverse-audit entry's verifier never ruled on it. Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
|
🤖 Addressed the latest review feedback (round 2/100). What changed, and what I pushed back on: · 已处理最新评审反馈（第 2/100 轮）。改动内容与我反驳保留之处如下： Review feedback round — PR #8498All 15 inline findings (1 Critical, 14 Suggestions) are resolved in code. Every Findings and dispositionsCritical
Suggestions — classifier hardening (
|
Summary
Stacked on #8468 (the reverse-audit budget gate) — merge that first; this PR's diff then shrinks to its own two commits.
Two independent measurements say the reverse-audit loop is where large-PR reviews spend most of their time: a +1699-line CI review ran the loop to the 5-round cap and was killed at 240 minutes mid-verification (#8368), and a +974-line local benchmark completed in 144 minutes with 95 of them (66%) in the loop — 5 rounds × 6 chunks. The round-by-chunk yield grid of that benchmark:
Chunks 3 and 6 were dry all five rounds — a third of the loop's auditors re-certifying converged territory — while the findings all came from the hot chunks. And every round waited serially for its verification before the next began.
The design intent — converge within one run, so misses do not leak into another full review cycle — is preserved by both changes; what they remove is redundancy, not depth.
Change 1: per-chunk retirement with cold rotation (CLI-enforced)
From round 3,
agent-prompt --all-chunksreads the harness transcripts (the same evidencecheck-coveragetrusts) and retires a chunk whose own last two audits were substantively dry — the receipt named what it examined AND the transcript shows the diff was opened; a whiff or a missing transcript keeps a chunk hot (fail toward auditing). A retired chunk is cold-checked on alternating rounds and returns to every-round auditing the moment a cold check yields. Rounds 1-2 always audit every chunk. Hot chunks get no less auditing than today; a late find in a retired chunk arrives one round later instead of never. (One bounded residual: a chunk retired late enough that its next cold check falls past the 5-round cap — e.g. retired after round 4 — is last audited at its retirement round; that tail is capped by the same hard limit that already ends the loop, and the chunk still holds the two-dry certificate the loop used to end on.) When every chunk holds the two-dry certificate and none is due, the builder printsCONVERGEDand exits 5 — a clean convergence: nounreviewedDimensionsentry, the per-chunk certificates being collectively the same evidence the global two-dry-rounds rule demanded. The budget gate (exit 4) runs first and is unchanged; single--chunkrebuilds bypass scheduling entirely; any classification error falls back to the full fan-out.Change 2: verification pipelines with the next round (SKILL)
Round k's verifiers and round k+1's auditors now launch in one response. The cumulative list exists so auditors do not re-report what is filed — an unverified entry serves that purpose exactly as well as a confirmed one, so the 5-8 minutes per round spent waiting for verdicts bought nothing. The orderings that matter stay: the last round's verification completes before Step 6, nothing unverified reaches the report or the PR, and an entry the verifier rejects leaves the list at the next merge.
Projected on the benchmark: the 95-minute loop drops to roughly 55 minutes (retirement −20% of auditors on that grid; pipelining −5-8 min/round), with detection semantics intact.
Test plan
9 new scheduling tests (rounds 1-2 full; round-3 skip with named note; cold-check parity; yield-returns-to-hot; whiff stays hot; all-retired → exit 5 with no records and no stamp; cold-check-only round still stamps;
--chunkbypass; transcripts-unavailable fallback; budget-gate-before-retirement pin). Fullsrc/commands/review: 1710 passed, 0 failed; eslint/prettier clean. Coverage interplay verified: skipped chunk-rounds write no records, and the Step 4/5 delivery floor is satisfied by rounds 1-2's records.中文说明
概述
叠在 #8468 之上——请先合并那个,本 PR 的 diff 随之缩为自己的两个提交。
两次独立测量都表明反审循环是大 PR 评审的最大时间去向:+1699 行的 CI 评审跑满 5 轮上限、在第 240 分钟被杀于验证中途(#8368);+974 行的本地基准 144 分钟完整跑完,其中 95 分钟(66%)在循环里——5 轮 × 6 chunk。逐轮×逐 chunk 产出网格见上(S=1 条 Suggestion,·=干净):chunk 3、6 五轮全干净——三分之一的 auditor 在反复认证早已收敛的领地;所有发现都来自热区。且每轮都串行等完验证才开下一轮。
设计初衷——在单次运行内收敛、避免漏检泄漏到下一整轮评审——两处改动都完整保留;砍掉的是冗余,不是深度。
变更 1:按 chunk 退休 + 冷轮换(CLI 强制)
第 3 轮起,
agent-prompt --all-chunks读取 harness transcripts(与check-coverage同源的证据),退休自身最近两次审计均"实质干净"的 chunk——回执点名检查对象且 transcript 显示确实打开过 diff;whiff 或缺 transcript 一律保持热区(失败方向=多审计)。退休 chunk 隔轮冷抽查,一旦出货立即回到每轮必审。第 1、2 轮永远全量。热区得到的审计不少于现状;退休区的晚期发现最多晚一轮到达,不会丢失。(一个有界残余:退休得足够晚、下次冷抽查落在 5 轮上限之外的 chunk——例如第 4 轮后退休——其最后一次审计即退休轮;这条尾巴被本就终结循环的硬上限所限,且该 chunk 仍持有循环过去赖以终止的"连续两轮干净"证书。)全部 chunk 持有"连续两轮干净"证书且本轮无人到期时,builder 打印CONVERGED并以 5 退出——干净收敛:不欠unreviewedDimensions条目,按 chunk 的证书合起来正是全局两干净轮规则所要求的同一份证据。预算闸(exit 4)先行且不变;单--chunk重建完全绕过调度;分类异常回退全量扇出。变更 2:验证与下一轮流水线化(SKILL)
第 k 轮的 verifier 与第 k+1 轮的 auditor 在同一个响应中发射。累计清单的作用是让 auditor 不重复报告——未验证条目与已确认条目在这个用途上等效,过去每轮 5~8 分钟的串行等待买不来任何东西。有意义的顺序保留:末轮验证在 Step 6 前完成,未验证内容绝不进报告或 PR,被 verifier 否决的条目在下次合并时移出清单。
按基准折算:95 分钟循环预计降至约 55 分钟(退休 −20% auditor;流水线每轮 −5~8 分钟),检测语义不变。
测试计划
9 个新调度用例(1-2 轮全量;第 3 轮跳过并点名;冷抽查奇偶;出货回热;whiff 保持热;全退休 → exit 5 且零记录零 stamp;纯冷查轮照常 stamp;
--chunk绕过;transcripts 不可用回退;预算闸先于退休)。全src/commands/review:1710 通过 0 失败;eslint/prettier 干净。覆盖率交互已核验:被跳过的 chunk-轮不写记录,Step 4/5 投递下限由 1-2 轮的记录满足。