`wandering` loop detector is outcome-blind

[gokcanakalin]

proof-4-environment-config-redacted.md
proof-1-session-transcript-redacted.md
proof-3-repro.test.txt
proof-2-repro-script.txt

atomic-agent 0.6.1 (installed standalone binary), stock config — no ATOMIC_AGENT_LOOP_* env vars, no loop* keys in config.json.
Affected code: src/agent/loop-detector.ts (ToolLoopTracker, formatForcedLoopReply), src/agent/batch-executor.ts (runSyncLoopGate), src/agent/agent-loop.ts (breaker path).
I'm attaching a redacted reproduction bundle (four files, see §9): session transcript, two runnable repros, environment/config proof.

---

1. Summary

I asked the agent a research question. Over one turn it made 12 distinct os.web.fetch calls — 11 of which returned status=ok with real content — and on the 12th call the wandering detector vetoed it and force-ended the turn with a synthetic reply telling me the agent was "stuck in a no-progress loop … after 12 blocked attempts".

Every part of that message is contradicted by the session record:

The reply said	The session record shows
"no-progress loop"	11/12 calls returned new, distinct, relevant content — that is progress
"12 blocked attempts"	exactly 1 call was blocked (the 12th); the other 11 executed
"still no answer"	the primary evidence for my question had already been retrieved at call #4

Having dug into the source, I found the root cause has two parts:

1. The wandering detector counts the spread of distinct arguments in a rolling window and never looks at whether those calls succeeded or whether the run was converging. Twelve distinct successful fetches and twelve dead-end URL guesses are indistinguishable to it.
2. The veto text (formatLoopGuidance) was written carefully enough to say "12 different attempts"; the forced reply (formatForcedLoopReply) was not — it hardcodes "no-progress loop" and "blocked attempts" for every breaker source, including wandering.

---

2. Environment

• atomic-agent 0.6.1, standalone binary at ~/.local/bin/atomic-agent, linux/x64
• Provider/model: OpenRouter, deepseek/deepseek-v4.1-flash (cloud, native tool calls)
• Loop config overrides: none — no ATOMIC_AGENT_LOOP_* env vars, no loop* keys in config.json
• Effective thresholds: the defaults, LOOP_WANDERING_THRESHOLD = 6 / LOOP_WANDERING_ESCALATION = 12

So nothing on my side tuned the detector — this is stock-config behaviour.

---

3. What happened

My question for the agent was: "Is the strict: true tool-calling issue for Inception Labs Mercury fixed in the latest atomic-agent?" It read the repo, the release notes, and then fetched a series of source files from raw.githubusercontent.com to confirm the fix in code.

3.1 Tool-call census for the turn

stepCount = 21   turnCount = 1   turns = 98   lastError = None
total tool calls = 48
counts = { os.fs.grep: 17, os.web.fetch: 12, os.shell.run: 6,
           os.fs.read: 5, os.git.log: 3, os.web.search: 2,
           os.fs.list: 2, os.fs.glob: 1 }

3.2 The 12 os.web.fetch calls, in order

#	status	URL (host raw.githubusercontent.com unless noted)
1	ok	github.com/AtomicBot-ai/atomic-agent/releases
2	ok	.../main/src/llm/provider/openai/openai-tool-call-adapter.ts
3	ok	.../main/src/llm/provider/openai/openai-build-body.ts
4	ok	.../main/src/llm/provider/openai/openai-strict-tools.ts
5	ok	.../main/src/llm/provider/registry/provider-types.ts
6	ok	.../main/src/config/config-schema.ts
7	ok	.../main/src/llm/provider/registry/provider-types.ts (re-read; maxChars 20000)
8	ok	.../main/src/llm/provider/openai/openai-strict-tools.ts (re-read; maxChars 20000)
9	ok	.../main/src/llm/provider/adapters/tool-call-adapter.ts
10	ok	.../v0.6.1/src/llm/provider/openai/openai-strict-tools.ts
11	ok	.../main/src/llm/provider/openai/openai-provider.ts
12	error	.../main/src/config/llm-config.ts ← the only blocked call

fetch_calls = 12, distinct_urls = 10, statuses = { ok: 11, error: 1 }.

One detail worth spelling out, because the numbers look off otherwise: the veto said "12 different attempts", but the table has only 10 distinct URLs. The reason is that hashToolCall (loop-detector.ts:675) hashes all arguments, not just the URL. Calls #7 and #8 re-fetched URLs already fetched (#5, #4) but with maxChars: 20000 instead of 12000 — different args, different argsHash, so they still count toward the spread. All 11 executed calls therefore contributed 11 distinct arg-hashes, and the prospective 12th call added one more: 11 + 1 = 12 ≥ 12 → escalation. (Reproduced deterministically in proof-2-repro-script.ts, scenarios B1/B2; full argument table in proof-1-session-transcript-redacted.md §2.)

3.3 The synthetic veto tool-result (call #12)

{
  "kind": "tool_result",
  "tool": "os.web.fetch",
  "status": "error",
  "summary": "BLOCKED: `os.web.fetch` — 12 different attempts against `raw.githubusercontent.com` and still no answer.\nChange strategy BEFORE calling any tool again:\n- Stop guessing URLs on `raw.githubusercontent.com`. Run `os.web.search` for the fact you need and fetch a result from a DIFFERENT host.\n- If you have enough information already, end the turn with `reply`.\n- Do NOT repeat this exact … [truncated]",
  "truncated": true
}

3.4 The forced terminal reply (what I actually saw)

(stopped: stuck in a no-progress loop on `os.web.fetch` after 12 blocked attempts).
I could not make further progress with the repeated tool call.
Here is my best answer with the information gathered so far — the task may be incomplete.

Note the contradiction between §3.3 and §3.4: the veto text correctly says "12 different attempts" (a spread of distinct arguments), while the forced reply says "12 blocked attempts" (implying 12 vetoes) and "no-progress loop" (implying repetition). Same event, two code paths, only one of them worded correctly. (Verbatim stored records: proof-1-session-transcript-redacted.md §3.5–§3.6.)

---

4. Root cause

4.1 The detector counts distinct arguments, not outcomes

src/agent/loop-detector.ts:

// loop-detector.ts:153 (v0.6.1)
export function isWanderingProneTool(tool: string): boolean {
  return (
    tool === "os.web.fetch" ||
    tool === "os.web.search" ||
    tool === "os.http.request" ||
    tool.startsWith("browser.")
  );
}

Worth quoting what this detector is for, because the design is sound and a fix should preserve it — the docstring directly above this function (lines 145–152 at v0.6.1):

"os.web.search is included: GAIA traces show small models burn an entire step budget re-formulating ~35 distinct search queries (different quotes / keywords / versions) while barely fetching the pages they already found. Each query is unique, so the args-only and no-progress streaks never fire — only the distinct-spread wandering detector can bound that token burn."

So the motivating pathology is search churn, and there the spread counter is doing a necessary job. The casualty in my incident is the other tool in that list — sequential multi-file os.web.fetch, a first-class workflow whose normal shape is every call succeeding. That asymmetry matters for the fix: search churn is successful-by-nature (distinct queries return distinct SERPs), while fetch wandering is failure-signed (dead-URL guessing surfaces as errors/404s). §7.2 leans on this.

// loop-detector.ts:325
isWanderingEscalated(tool: string, args: unknown): boolean {
  if (!isWanderingProneTool(tool)) return false;
  const argsHash = hashToolCall(tool, args);
  return this.effectiveSpread(tool, argsHash) >= this.wanderingEscalation;
}

// loop-detector.ts:336
private effectiveSpread(tool: string, currentArgsHash: string): number {
  const seen = new Set<string>();
  for (const record of this.history) {
    if (record.tool !== tool) continue;
    if (typeof record.resultHash !== "string" || !record.resultHash) continue;
    seen.add(record.argsHash);
  }
  return seen.has(currentArgsHash) ? seen.size : seen.size + 1;
}

effectiveSpread counts distinct argsHash values in the window. It does not look at:

• whether each call returned status: "ok" or status: "error";
• whether the returned content was new/relevant or a duplicate;
• whether the run was converging (each fetch narrowing the answer) or diverging.

A resultHash merely has to exist — a successful fetch and a failed fetch count toward the spread identically (hashToolOutcome, loop-detector.ts:685, collapses errors to a stable error:<hash> just as it hashes successful bodies). So "12 distinct successful, relevant fetches" and "12 distinct dead-end URLs" are indistinguishable to this detector. Relatedly, because hashToolCall (loop-detector.ts:675) hashes all args, the same URL fetched with a different maxChars counts as a new spread entry — which is how my turn's spread reached 12 with only 10 distinct URLs (see §3.2).

4.2 The escalation rides the breaker path

src/agent/batch-executor.ts, runSyncLoopGate (batch-executor.ts:611, v0.6.1):

const breakerTripped = ctx.tracker.isBreakerTripped(tool, args);
const wanderingEscalated = ctx.tracker.isWanderingEscalated(tool, args);
const verdict = ctx.tracker.check(tool, args);
ctx.tracker.recordCall(tool, args);

if (verdict.level === "critical" || breakerTripped || wanderingEscalated) {
  const forceBreaker = breakerTripped || wanderingEscalated;
  ...
  const detector =
    wanderingEscalated && verdict.detector === "wandering"
      ? "wandering"
      : verdict.detector;
  const vetoResult = compressToolResult({
    tool, status: "error",
    output: formatVetoInstruction({ tool, count, target, detector }),
    ...
  });
  ...
  loopSignals.push({ kind: forceBreaker ? "breaker" : "critical", tool, count, detector, ... });
  return { proceed: false, vetoResult };
}

The code is careful here: it passes detector through so the veto wording can say "different attempts" instead of "identical outcomes" (see the comment at 637–647). That is exactly why §3.3 reads correctly.

4.3 The forced reply drops the detector

src/agent/agent-loop.ts (agent-loop.ts:1442–1444, v0.6.1):

const breaker = loopSignals.find((s) => s.kind === "breaker");
if (breaker) {
  const replyText = formatForcedLoopReply(breaker.tool, breaker.count);
  ...
}

formatForcedLoopReply has the signature (tool, count) — the detector field that was so carefully threaded into the veto path is not passed here (a few lines later the same breaker.detector is used for the loop_detected event, so it's available). The forced reply therefore cannot know it came from a wandering spread:

// src/agent/loop-detector.ts:1047 (v0.6.1)
export function formatForcedLoopReply(tool: string, count: number): string {
  return [
    `(stopped: stuck in a no-progress loop on \`${tool}\` after ${count} blocked attempts).`,
    "I could not make further progress with the repeated tool call.",
    "Here is my best answer with the information gathered so far — the task may be incomplete.",
  ].join(" ");
}

Three false statements, all hardcoded:

1. "no-progress loop" — wrong for a wandering spread (which is by definition distinct arguments, and here 11/12 succeeded).
2. "12 blocked attempts" — count here is the argument spread (12), not the number of vetoes (1). The veto path already knows this and words it as "12 different attempts"; the forced reply reuses the same number with the wrong noun.
3. "I could not make further progress with the repeated tool call" — there was no repeated tool call, and progress was being made.

4.4 "Still no answer" was also false

The 4th fetch returned the header of openai-strict-tools.ts, which opens with:

"Some models call tools reliably only when the provider constrains decoding to the function's parameters schema. That is what OpenAI's strict mode does, and models built for it (Inception Labs' Mercury was the report that prompted this) produce a stream of malformed calls without it."

So the primary evidence for my question had already been retrieved at call #4, eight calls before the detector fired. The stored reasoning on the (blocked) 12th call even opens with "I've found strong evidence … So yes, fixed" (verbatim record: proof-1-session-transcript-redacted.md §3.4). "Still no answer" was false at the moment it was printed.

---

5. Why this matters

1. The user is shown a false diagnosis. The turn ends with a message that contradicts the session record. Reading it, you would conclude the agent was stuck in a loop; it was not.
2. Legitimate sequential retrieval is penalised. The detector was built to bound search churn (the GAIA docstring quoted in §4.1); the workflow it cut off here is sequential multi-file os.web.fetch — normal, converging work whose every call succeeded. The two churns have different signatures (§4.1), and the detector can't tell them apart.
3. The turn is terminated early. The breaker path forces a synthetic reply and ends the turn — so the agent cannot finish the answer it was assembling.
4. The wording bug is a one-line fix, but the detector blindness is the real issue. Even with correct wording, an outcome-blind spread counter will still cut off converging multi-fetch runs.

---

6. Reproducing

6.1 Quick unit repro (deterministic, no network needed)

The detector is pure and unit-testable. Driving ToolLoopTracker directly reproduces the misclassification:

import { ToolLoopTracker } from "../src/agent/loop-detector.js";

const tracker = new ToolLoopTracker(); // defaults: wanderingEscalation = 12

// 11 DISTINCT, SUCCESSFUL fetches (each returns real content).
// Runtime protocol (runSyncLoopGate): check -> recordCall -> recordOutcome.
for (let i = 0; i < 11; i++) {
  const args = { url: `https://example.com/file-${i}.ts` };
  tracker.check("os.web.fetch", args);
  tracker.recordCall("os.web.fetch", args);
  tracker.recordOutcome("os.web.fetch", args, {
    tool: "os.web.fetch",
    status: "ok",
    summary: `content of file ${i}`,   // distinct, successful (CompressedToolResult shape)
    details: {},
  });
}

// The 12th distinct, successful fetch:
const args12 = { url: "https://example.com/file-11.ts" };
console.log(tracker.isWanderingEscalated("os.web.fetch", args12)); // => true

Current behaviour: true — the turn is escalated even though every call succeeded. Desired: false — 12 distinct successful fetches are converging work, not a wandering loop.

I've attached turnkey versions of this (see §9): proof-2-repro-script.ts (run with npx tsx; four scenarios, including the session-exact argument sequence and an arg-identical control) and proof-3-repro.test.ts (vitest; four passing tests that encode current behaviour — flip the expectations when the fix lands). Output I get on the current tree:

A1  12 distinct args, 11 recorded so far, ALL successful  -> isWanderingEscalated = true
A2  control — verbatim repeat of call #1 args            -> isWanderingEscalated = false
B1  session-exact: 11 executed (10 URLs, all ok) + 12th  -> isWanderingEscalated = true
B2  same, but re-reads arg-identical (9 distinct hashes) -> isWanderingEscalated = false

6.2 End-to-end reproduction

1. Run a turn whose task requires reading several distinct source files from one host (e.g. "explain how feature X is implemented across these modules").
2. Let the agent make ≥ 12 distinct os.web.fetch calls to that host, all returning status: ok.
3. Observe: the 12th call is vetoed with BLOCKED: … N different attempts against <host> and still no answer, and the turn ends with (stopped: stuck in a no-progress loop on \os.web.fetch` after 12 blocked attempts)`.

6.3 How I extracted the session evidence

The session store is the ground truth: ~/.atomic-agent/sessions.sqlite, table sessions(id, working_dir, status, payload, created_at, updated_at), where payload is the JSON session state (turns[] with kind ∈ {user, assistant_tool_call, tool_result, assistant_reply}).

sqlite3 ~/.atomic-agent/sessions.sqlite \
  "select id, datetime(updated_at/1000,'unixepoch'), length(payload)
     from sessions where payload like '%stuck in a no-progress loop%'
     order by updated_at desc limit 5;"

Then dump the payload and pair each assistant_tool_call with the following tool_result by tool name to get per-call status and summary. The counts in §3.1–§3.2 come from exactly that pairing.

---

7. Suggested fixes

7.1 Minimum (wording) — pass the detector into the forced reply

// agent-loop.ts
const replyText = formatForcedLoopReply(breaker.tool, breaker.count, breaker.detector);

// loop-detector.ts
export function formatForcedLoopReply(
  tool: string,
  count: number,
  detector?: LoopCheckVerdict["detector"],
): string {
  if (detector === "wandering") {
    return [
      `(stopped: \`${tool}\` was called with ${count} different arguments this turn without converging).`,
      "I could not settle on the answer with the pages I fetched.",
      "Here is my best answer with the information gathered so far — the task may be incomplete.",
    ].join(" ");
  }
  return [ /* existing text */ ].join(" ");
}

This removes all three false statements for the wandering case and costs nothing for the repeat case.

7.2 Substantive (detector) — an outcome-aware spread, scoped to fetch

The docstring quoted in §4.1 makes the design intent explicit: this detector exists to bound search churn. That intent should be protected, which is why the outcome-awareness needs to be scoped, not blanket:

• Apply outcome-awareness to os.web.fetch (and os.http.request) only. On fetch, the behaviours the detector wants to separate are objectively distinguishable: dead-URL guessing yields non-ok outcomes (errors, 404s, empty bodies); sequential multi-file retrieval yields ok with distinct content. Count only the former toward the spread — effectiveSpread already walks history and has resultHash, it just needs the status too.
• Exclude duplicate content within fetch. Two calls returning byte-identical bodies are one attempt, not two. (Calls feat: AIML API integration & Structured Outputs support #7 and Feature/searchfix #8 in §3.2 re-fetched already-fetched URLs — with a different maxChars, so the args-hash spread counted them; under a duplicate-content rule they would not.)
• Leave the os.web.search spread as it is (or give fetch its own, higher threshold). A blanket "count only non-ok outcomes" — or a convergence signal where "new content ⇒ converging" — applied to all wandering-prone tools would regress the documented case: ~35 distinct successful queries, each returning a distinct SERP (hence always "new content"), would never escalate, and the token-burn bound would be gone. Search churn is successful-by-nature; only fetch churn is failure-signed.

7.3 Complementary — surface the escalation as a notice, not a turn-ending breaker

Independent of the above, and low-risk on its own: a wandering escalation could inject the (already well-written) formatWanderingRedirect notice (loop-detector.ts:1022) and let the model decide, reserving the forced reply for the true repeat case (breakerTripped). This keeps the warn-and-redirect nudge at spread 6, removes both the false terminal reply and the premature turn-end, and leaves the "should anything hard-stop a burn" question to the scoped fix in §7.2.

---

8. Evidence appendix (redacted)

Privacy redactions applied: session id, working directory, user identity, and any local absolute paths are replaced with placeholders. No user content beyond the tool-call metadata is reproduced. The fuller version of this evidence (verbatim records incl. full fetch arguments and the model's reasoning on the blocked call) is proof-1-session-transcript-redacted.md.

session_id      = <REDACTED>
working_dir     = <REDACTED>
stepCount       = 21
turnCount       = 1
turns           = 98
lastError       = None
total_tool_calls= 48
counts          = {os.fs.grep: 17, os.git.log: 3, os.fs.glob: 1, os.fs.read: 5,
                   os.fs.list: 2, os.shell.run: 6, os.web.search: 2, os.web.fetch: 12}
fetch_calls     = 12
distinct_urls   = 10
statuses        = {ok: 11, error: 1}

 1 idx=69 status=ok    url=https://github.com/AtomicBot-ai/atomic-agent/releases
 2 idx=71 status=ok    url=https://raw.githubusercontent.com/.../openai-tool-call-adapter.ts
 3 idx=73 status=ok    url=https://raw.githubusercontent.com/.../openai-build-body.ts
 4 idx=77 status=ok    url=https://raw.githubusercontent.com/.../openai-strict-tools.ts
 5 idx=79 status=ok    url=https://raw.githubusercontent.com/.../provider-types.ts
 6 idx=81 status=ok    url=https://raw.githubusercontent.com/.../config-schema.ts
 7 idx=83 status=ok    url=https://raw.githubusercontent.com/.../provider-types.ts
 8 idx=85 status=ok    url=https://raw.githubusercontent.com/.../openai-strict-tools.ts
 9 idx=87 status=ok    url=https://raw.githubusercontent.com/.../tool-call-adapter.ts
10 idx=89 status=ok    url=https://raw.githubusercontent.com/.../v0.6.1/.../openai-strict-tools.ts
11 idx=91 status=ok    url=https://raw.githubusercontent.com/.../openai-provider.ts
12 idx=95 status=error url=https://raw.githubusercontent.com/.../config/llm-config.ts

--- error tool_results (full) ---
[1] os.git.log  status=error
    "git --no-pager log ... exited with 128: fatal: not a git repository"
    (unrelated to this issue — the repo path given was a parent dir)

[2] os.web.fetch status=error
    "BLOCKED: `os.web.fetch` — 12 different attempts against `raw.githubusercontent.com`
     and still no answer.
     Change strategy BEFORE calling any tool again:
     - Stop guessing URLs on `raw.githubusercontent.com`. Run `os.web.search` for the fact
       you need and fetch a result from a DIFFERENT host.
     - If you have enough information already, end the turn with `reply`.
     - Do NOT repeat this exact … [truncated]"

--- final assistant_reply ---
"(stopped: stuck in a no-progress loop on `os.web.fetch` after 12 blocked attempts).
 I could not make further progress with the repeated tool call.
 Here is my best answer with the information gathered so far — the task may be incomplete."

---

9. Attachments — reproduction bundle & fix checklist

All four files are redacted (no session ids, local paths, user content, or credentials) and can be attached as-is. On GitHub the two .ts files go up as .txt — GitHub rejects .ts uploads:

File	What it shows	How to use
proof-1-session-transcript-redacted.md	Primary session evidence: full turn census, all 12 fetch arg-sets with statuses, the spread arithmetic (§3.2), verbatim veto / forced-reply / final-call records incl. the model's reasoning showing the answer was already in hand (§4.4)	read; extraction method in §6.3 / proof-1 §0
proof-2-repro-script.ts	Deterministic detector repro — no network, no config, no DB. Four scenarios: minimal (§6.1), repeat control, session-exact args, arg-identical control	copy to repo root → npx tsx proof-2-repro-script.ts (expected output in file header)
proof-3-repro.test.ts	Same four scenarios as a vitest suite asserting current behaviour (passes on 0.6.1; flip expectations for the fix)	copy to src/agent/issue-repro-wandering-outcome-blind.test.ts → npx vitest run src/agent/issue-repro-wandering-outcome-blind.test.ts
proof-4-environment-config-redacted.md	Stock-config proof: version/platform, zero loop overrides (env + config), defaults with source refs, redacted provider config, and confirmation the quoted code is in the shipped 0.6.1 binary	read

Fix checklist (summarising §7):

• formatForcedLoopReply accepts and honours detector (wording fix, §7.1)
• effectiveSpread stops counting successful, content-returning fetches as "wandering" (§7.2)
• Duplicate-content fetches do not inflate the spread (§7.2)
• The fix does not regress the documented search-churn case — ~35 distinct successful searches must still escalate (§4.1, §7.2)
• Consider demoting wandering escalation from breaker to notice (§7.3)
• Unit test: 12 distinct successful fetches must not escalate (§6.1)

[sosidudku1]

Thank you for this report. The reproduction bundle, and especially the unit repro against ToolLoopTracker, made the problem clear in minutes.

Part of it is fixed. #460, shipped in v0.6.3, passes the detector into the forced reply, so a stop by the wandering detector no longer claims a "no-progress loop" or "12 blocked attempts". It now says the tool hit the limit on different arguments and that the last call was not run. v0.6.4 also corrected the refusal count quoted in the other stop messages. atomic-agent update gets you both.

The detector itself is unchanged: effectiveSpread still counts distinct arguments without looking at outcomes. Making it outcome-aware for os.web.fetch (your §7.2) or turning the escalation into a notice (§7.3) is still an open design question. The reason we haven't simply counted non-ok results is that status: ok doesn't prove a fetched page was useful. Keeping this open for that part.