Skip to content

fix(ios): report an unobserved text commit instead of a partial success - #1924

Merged
thymikee merged 6 commits into
mainfrom
fix/1874-synthesized-commit-timeout
Aug 21, 2026
Merged

fix(ios): report an unobserved text commit instead of a partial success#1924
thymikee merged 6 commits into
mainfrom
fix/1874-synthesized-commit-timeout

Conversation

@thymikee

@thymikee thymikee commented Aug 20, 2026

Copy link
Copy Markdown
Member

Refs #1874, #1844.

Summary

Stops the iOS synthesized bare-type route from reporting success when the app has not observably committed all requested text.

  • Distinguishes a settled commit from a strict-prefix wait that reaches its three-second deadline. The latter now returns TEXT_INPUT_COMMIT_NOT_OBSERVED with recovery guidance to read the field and prefer fill over appending another type.
  • Observes once more before condemning the final polling interval, so a commit that lands during the last sleep is not falsely rejected.
  • Treats a final value equal to the field placeholder conservatively. Accessibility cannot distinguish committed text from an empty field rendering that placeholder—even when the field held content before dispatch, because an input handler may clear it while processing the event.
  • Reuses the existing placeholder-normalization predicate instead of maintaining a second equality rule, and removes the superseded baseline-dependent evidence enum/raw-read path.
  • Keeps submit-key and unreadable-field behavior unchanged (.unobservable remains non-failing).

This fixes #1874's strict-prefix signature ("h" remaining from "hardware-keyboard"). It does not claim to fix #1844's reordered/non-prefix text, which follows the existing transformed-input path. The sibling synthesized replacement and keyboard-visible routes remain outside this observation mechanism because they do not retain a resolvable element.

Validation

  • Red-before on the clear-after-dispatch regression: the old non-empty-baseline model returned no failure when the post-dispatch AX value was the placeholder (nil vs TEXT_INPUT_COMMIT_NOT_OBSERVED).
  • Green on the existing iPhone 17 Pro simulator after the fix: the clear-after-dispatch regression and the nearest ordinary-placeholder negative case both passed (2 tests, 0 failures). Eight adjacent commit-policy tests also passed.
  • Removing the conservative guard made the regression fail again with two assertions: missing TEXT_INPUT_COMMIT_NOT_OBSERVED and one forbidden post-dispatch observation.
  • The focused iOS runner build completed with TEST BUILD SUCCEEDED; pnpm check:xctest-selection reports 185 declared tests and 0 unreachable.
  • pnpm check:affected --run passed at a58410266; GitHub-authoritative Swift, coverage, and device lanes are running on that exact head.

Docs were updated because this changes when TEXT_INPUT_COMMIT_NOT_OBSERVED is returned and how callers should recover. No skill guidance changed; the command reference owns this behavior.

@github-actions

github-actions Bot commented Aug 20, 2026

Copy link
Copy Markdown
PR Preview Action v1.8.1
Preview removed because the pull request was closed.
2026-08-21 13:13 UTC

@github-actions

github-actions Bot commented Aug 20, 2026

Copy link
Copy Markdown

Size Report

Metric Base Current Diff
JS raw 2.36 MB 2.36 MB 0 B
JS gzip 789.9 kB 789.9 kB 0 B
npm tarball 906.2 kB 907.5 kB +1.3 kB
npm unpacked 3.14 MB 3.15 MB +4.0 kB

Startup median (7 runs, lower is better):

Scenario Base Current Diff
CLI --version 28.4 ms 28.9 ms +0.5 ms
CLI --help 83.5 ms 83.4 ms -0.1 ms

Top changed chunks: no changes in the largest emitted chunks.

@thymikee

Copy link
Copy Markdown
Member Author

Adversarial review pass (Fable). No blocker, but two deterministic false-failure windows in my own wait, one overclaim in the PR framing, and an untested branch. All addressed in 03a0a45b6; the review also refuted several things I want on the record.

Fixed

1. The deadline was checked before observing. A commit landing during the final poll sleep was condemned as .notObserved — under exactly the loaded-host timing this wait exists for, so the fix could have added red to the lane it came from. The check now runs after an observation, so the last thing before condemning is a read. testCommitLandingDuringTheFinalSleepIsStillObserved is red against the ordering I first pushed (verified by running both loop structures side by side: old → notObserved, new → settled).

2. Placeholder collision. treatingPlaceholderAsEmpty maps a value equal to the field's placeholder to "", which is a prefix of every expected value. type "0.00" into a field placeheld "0.00" committed instantly, read as pending for the full 3s, then failed. The observation now settles on an exact raw match; the normalized read still drives the prefix walk. Same trap for "Search"/"Search" and the generic "Text input field" label.

3. The refusal branch was untested. The whole fix was one ternary that no test reached. It is now textEntryFailure(forCommitOutcome:) with assertions on all three cases — including .unobservable → nil, which is what keeps type "...\n" working and would have failed silently if inverted.

4. Message asserted a field state never read. Under both windows above the field may hold the complete value, so "committed only part" was wrong. Now "could not confirm the typed text committed before its deadline".

Corrected framing — this does not catch #1844's signatures

I overclaimed. Both #1844 values classify as .diverged, not .pending, so they settle and still return ok:

observed expected class
.test ada@example.test diverged → settles, stays ok
adexamplea@.test ada@example.test diverged → settles, stays ok
h hardware-keyboard pending → now fails

Only the strict-prefix stall (#1874's signature) becomes a failure. #1844's lost/reordered prefix is a different mechanism and this PR does not address it. The docs sentence has been narrowed to match, and no longer implies every text-entry route verifies its result.

Known, unchanged

  • The sibling routes still return verified: nil without observing: runSynthesizedReplacementRoute, and the keyboard-visible synthesized branch. Neither has a resolvable element to observe, so the mechanism cannot apply as-is — noted rather than silently left.
  • Paced typing (--delay-ms) can now abort mid-string. The commit wait runs per character, so a >3s stall on character 1 fails the command with characters 2..n never posted — where before every character was posted and a slow app eventually drained the queue. I believe failing is right (the alternative is a success over an unconfirmed field), but it is a real semantics change on that route and was unstated. Worst-case latency is unchanged; the per-character wait predates this PR.

Refuted — checked and sound

  • The dispatched deletion. All seven pre-change return paths enumerated including the #else/macOS branches: dispatched was true iff failure == nil everywhere, and .raise never returns. The four ?? .notFocused fallbacks were dead code.
  • Secure fieldseditableTextValue nil → textBefore nil → .unobservable → no failure, exempt by construction.
  • Elements that stop resolving mid-wait → nil → .diverged → settles.
  • Formatters / autocomplete / card spacing → non-prefix → diverges immediately.
  • A maxLength field that truncates does now fail where it used to succeed — but truthfully, since the field genuinely lacks the requested text.
  • Error-code registration: runner codes pass through ErrorPayload generically; the two sibling codes have zero TypeScript references, so there is no registry to miss. ADR-0010 needs code+message+hint, all present.
  • The hint's fill advice: fill uses replacement mode, never enters the fixed branch, and surfaces a dropped character as TEXT_ENTRY_MISMATCH.

pnpm gate swift-runner-macos builds clean; pnpm check:affected --run passes; check:xctest-selection now reports 166 methods, 143 host-lane-reachable, 0 unreachable.

@thymikee

Copy link
Copy Markdown
Member Author

[P1] Make the placeholder regression cover the production normalization boundary. testValueEqualToThePlaceholder… injects an observe closure that already returns "0.00", so removing the new raw-value short-circuit in awaitSynthesizedFirstResponderCommit leaves this test green—the test never supplies the normalized "" that causes the failure. Extract the raw-exact/normalized-prefix choice into a small pure helper and test it with (raw: "0.00", normalized: "", expected: "0.00"), or exercise it with a fixture element. That way reverting the fix goes red.

@thymikee

Copy link
Copy Markdown
Member Author

[P1] addressed in 40c70f51e. You were right, and the failure mode was worse than "weak test": I had claimed in the review-response comment above that it pinned the behavior, and it could not.

Confirmed first. The fix lived at RunnerTests+SynthesizedTextEntry.swift:271 inside the production observe closure; the test injected its own observe returning "0.00" and never reached that line. Reverting the short-circuit left it green.

Now. The raw-exact/normalized-prefix choice is commitObservation(rawValue:expectedText:normalizedValue:), driven with exactly the case you named:

XCTAssertEqual(
  Self.commitObservation(rawValue: "0.00", expectedText: "0.00", normalizedValue: { "" }),
  "0.00")

Red-before proved by running both implementations against the assertion set:

with the fix   : GREEN
fix reverted   : RED — exact-match case

Three more cases pin the other direction, so the helper cannot be "fixed" by always returning the raw value: a non-exact match keeps the normalized reading (raw "0.00" / expected "0.005"""), the prefix walk still works, and nil stays nil.

One deviation from your suggestion. normalizedValue is a closure, not a value. Taking both readings eagerly would make every poll cost two accessibility reads on a path that polls every 20ms for up to three seconds; the closure keeps an exact match at one read. testCommitObservationDoesNotPayForASecondReadOnAnExactMatch pins that, so the laziness cannot be refactored away silently. If you would rather have the simpler eager signature and accept the extra read, say so and I will flip it.

I did not take the fixture-element option — it would put this on the simulator lane, where the host lane now covers it in 12ms per PR.

The old test is deleted rather than kept alongside: its only other assertion (an exact match settles without polling) is already testSynthesizedCommitStopsAtTheFirstSettledObservation.

pnpm gate swift-runner-macos clean, pnpm check:affected --run passes, check:xctest-selection 167 declared / 144 host-reachable / 0 unreachable.

@thymikee

Copy link
Copy Markdown
Member Author

P1: Do not use placeholder equality as commit evidence.

editableTextValue(..., treatingPlaceholderAsEmpty: true) deliberately classifies a raw value equal to element.placeholderValue as empty. For an empty field whose placeholder is 0.00, textBefore is therefore ""; after dispatching type "0.00", the first read can still be the unchanged placeholder 0.00. commitObservation treats that raw equality as committed before normalization, so the wait settles and the command returns ok: true without evidence that any synthesized character committed. That reintroduces the success-misdescribes-device-state failure this PR is intended to remove.

Please keep this state indeterminate/non-success (or obtain a signal that distinguishes placeholder rendering from a committed value) and add a regression where the raw placeholder already equals the requested text before any commit; it must not report success.

@thymikee
thymikee force-pushed the fix/1874-synthesized-commit-timeout branch from 40c70f5 to 7cedfc5 Compare August 21, 2026 09:37
@thymikee

Copy link
Copy Markdown
Member Author

P1 addressed in 7cedfc59b; branch rebased onto 46eff36f8.

You are right, and the direction matters: my fix for a false failure created a false success, which is strictly worse and is the exact defect this PR exists to remove.

Confirmed the mechanism before changing anything. Empty field, placeholder 0.00, type "0.00":

textBefore (normalized) =           <- placeholder classified as empty
expectedText            = 0.00
first read, nothing committed: raw = 0.00   <- the placeholder rendering
commitObservation: raw == expected -> 0.00 -> .committed -> .settled -> ok:true
=> success reported with ZERO characters committed

No disambiguating signal exists. element.value is byte-identical whether the placeholder is rendering or the committed text happens to equal it, and placeholderValue only tells you what the placeholder is, not which of the two states you are in. So I took your first option rather than the second: the state stays non-success.

commitObservation is deleted, not narrowed. The raw match was only ever consulted in precisely this case, and in precisely this case it is not evidence — so there was nothing left worth keeping. The observation is the plain normalized read again, as before that revision.

The refusal is now up front, not after the deadline. Since no read can resolve the ambiguity, spending 3 seconds discovering that is pure latency. placeholderMakesCommitUnobservable detects it before the wait starts and reports .notObserved immediately.

Regression, as you asked — raw placeholder already equals the requested text before any commit, must not report success:

XCTAssertTrue(
  Self.placeholderMakesCommitUnobservable(placeholder: "0.00", expectedText: "0.00"))
XCTAssertEqual(
  Self.textEntryFailure(forCommitOutcome: .notObserved), .commitNotObserved)

Red-before proved against the previous revision on the zero-characters-committed scenario:

previous revision: ok:true (SUCCESS)
now:               TEXT_INPUT_COMMIT_NOT_OBSERVED

A second test keeps the guard narrow — placeholder "0.00" / expected "0.005", placeholder "Email" / expected "ada@example.test", nil, empty and whitespace-only placeholders all stay observable. Widening it would refuse ordinary typing into any placeheld field, which is most of them.

Two knock-on corrections. The failure message no longer says "before its deadline" (this refusal never waits), and the docs now name both routes to the code — deadline expiry, and placeholder-identical text.

The eager-vs-lazy question from my last comment is moot — the helper it applied to is gone.

Rebased onto 46eff36f8; verified line-level that nothing from main was reverted (git log origin/main ^HEAD empty, all deletions attributable to this branch). pnpm gate swift-runner-macos clean, pnpm check:affected --run passes, check:xctest-selection 184 declared / 161 host-reachable / 0 unreachable.

@thymikee

Copy link
Copy Markdown
Member Author

[P1] Keep the placeholder guard scoped to an empty baseline. placeholderMakesCommitUnobservable currently receives only the placeholder and final expected text, so it refuses any append whose resulting value happens to equal the placeholder. For example, with actual value "0" and placeholder "0.00", type ".00" computes expected "0.00" and returns TEXT_INPUT_COMMIT_NOT_OBSERVED before a read, although the non-empty textBefore proves a later "0.00" is commit evidence rather than the initial placeholder rendering. The indistinguishable state exists only when the normalized pre-dispatch value is empty. Gate this refusal on that baseline as well, and add a regression for "0" + ".00" that reaches the observation path and settles.

@thymikee

Copy link
Copy Markdown
Member Author

P1 addressed in afe99bd15.

Right again — I had made the guard baseline-blind, so it refused the very case where the read is trustworthy. Value "0" + type ".00" against placeholder "0.00" was refused before a single read.

The baseline is the deciding input, so it is now an input. placeholderCommitEvidence returns three states instead of a boolean:

state when behaviour
normalRead expected differs from the placeholder placeholder never enters into it
indistinguishable expected IS the placeholder and the field was empty refuse, before the wait
rawValueIsEvidence expected IS the placeholder but the field held content raw match settles it

Only .indistinguishable refuses. .rawValueIsEvidence reaches the observation and settles on the raw match, which the normalized read would otherwise hide — that is your "0" + ".00" case.

commitObservation is back for that third state only, scoped by evidence rather than applied unconditionally as in the revision that turned raw equality into a false success.

Both directions proved, so neither P1 regresses the other:

Case A — empty field, placeholder "0.00", type "0.00" (nothing committed):
   baseline-blind (previous): REFUSED before any read
   baseline-scoped (now):     REFUSED before any read     <- still correct

Case B — value "0", type ".00", placeholder "0.00" (commit DOES land):
   baseline-blind (previous): REFUSED before any read     <- the defect
   baseline-scoped (now):     settled -> ok

The regression you asked for reaches the observation path rather than asserting on the classifier alone: testAppendWhoseResultEqualsThePlaceholderSettlesThroughTheObservation drives awaitSynthesizedCommitOutcome with rawValue walking "0""0.00" and normalizedValue pinned at "" throughout. Because the normalized reading never matches, only the evidence-scoped raw match can settle it — so the test goes red both if the guard reverts to baseline-blind and if commitObservation ignores evidence.

testPlaceholderEvidenceDoesNotFireOnOrdinaryTyping now runs its cases under both baselines, so the narrowness is pinned independently of the new input. testOrdinaryObservationDoesNotPayForTheRawRead keeps normalRead at one accessibility read per poll.

Docs updated to name the empty-field condition rather than implying any placeholder-identical text refuses.

pnpm gate swift-runner-macos clean, pnpm check:affected --run passes, check:xctest-selection 187 declared / 164 host-reachable / 0 unreachable.

@thymikee

Copy link
Copy Markdown
Member Author

[P1] A non-empty baseline is not post-dispatch commit evidence. rawValueIsEvidence assumes that because the field held "0" before type ".00", a later raw "0.00" must be committed text rather than the placeholder. But an input handler can clear the field while processing the synthesized event; the now-empty field renders the same raw "0.00" placeholder, and commitObservation reports success over an empty field. The baseline proves only what rendered before dispatch, not what the later AX read represents. Add that clear-after-dispatch regression and require TEXT_INPUT_COMMIT_NOT_OBSERVED; without a post-dispatch discriminator, keep placeholder-equal reads conservative.

awaitSynthesizedFirstResponderCommit returned Void, so its three exits were
indistinguishable to the caller: the expected text committed, the app
transformed the input, or the 3s deadline expired with a strict prefix still
outstanding. The caller returned dispatched-with-no-failure in all three, and
`type` answered ok with textEntryRoute synthesized-first-responder over a field
holding part of the requested text.

The wait now returns a SynthesizedTextCommitOutcome and an expired deadline
becomes TEXT_INPUT_COMMIT_NOT_OBSERVED, whose hint points at fill rather than a
type retry — type appends, so retrying it concatenates onto whatever committed.

The tail is still not re-synthesized: #1676 rejected that because a stalled
prefix cannot be told apart from a suffix still queued, so repair double-posts.
Reporting is what the runner does instead.

typeIntoCurrentTarget loses its `dispatched` flag, which was exactly
`failure == nil` and could not express the new state — characters posted, commit
unconfirmed, command must refuse. Failure is now the single discriminator.

The decision moves behind an injected clock/observer so the deadline branch runs
in the macOS host lane on every PR instead of needing a simulator.

Refs #1874, #1844
Adversarial review found two deterministic false failures in the wait added by
the previous commit, plus a message that asserted a field state never read.

The deadline was checked before observing, so a commit landing during the final
poll sleep was condemned as never observed — under exactly the loaded-host
timing the wait exists for. The check now runs after an observation, so the last
thing before condemning is a read.

`treatingPlaceholderAsEmpty` maps a value equal to the field's placeholder to
"", a prefix of every expected value. `type "0.00"` into a field placeheld
"0.00" committed instantly, read as pending for the full 3s, and failed. The
observation now settles on an exact raw match; the normalized read still drives
the prefix walk.

The outcome-to-failure mapping moves to textEntryFailure(forCommitOutcome:) so
the branch the command refuses on is pinned by a test rather than living only in
a ternary. `.unobservable` staying a success is what keeps `type "...\n"`
working, and it now has an assertion.

Message and hint no longer claim the field holds a partial value: under both
fixed windows it may hold all of it. The docs sentence no longer implies every
text-entry route verifies its result — the replacement and keyboard-visible
routes have no resolvable element to observe and are unchanged.

Refs #1874, #1844
Review [P1]: testValueEqualToThePlaceholder… injected an observe closure that
already returned "0.00", so it never supplied the normalized "" that causes the
failure. The raw-value short-circuit lived in the production observe closure,
which that test bypassed entirely — reverting the fix left it green.

The raw-exact/normalized-prefix choice moves into commitObservation, and the
test drives it with (raw: "0.00", normalized: "", expected: "0.00"). Reverting
commitObservation to always return the normalized reading now fails the
exact-match assertion.

normalizedValue is a closure rather than a value so an exact match still costs
one accessibility read instead of two, on a path that polls every 20ms for up to
three seconds; a second test pins that laziness.

The old test is deleted rather than kept: its remaining assertion (an exact
match settles without polling) is already covered by
testSynthesizedCommitStopsAtTheFirstSettledObservation.

Refs #1874
Review [P1]: an empty text field renders its placeholder AS its accessibility
value, which is why editableTextValue(treatingPlaceholderAsEmpty:) classifies
that value as empty. The previous revision's raw-exact short-circuit therefore
matched BEFORE anything committed whenever the requested text was the
placeholder: `type "0.00"` into a field placeheld "0.00" settled on the first
read and returned ok with zero characters delivered — reintroducing the
success-misdescribes-the-device failure this PR exists to remove.

The state is structurally indeterminate. element.value is identical whether the
placeholder is rendering or the committed text happens to equal it, and
placeholderValue does not disambiguate, so no read resolves it and waiting the
deadline out discovers nothing. placeholderMakesCommitUnobservable detects it up
front and reports the commit unobserved, which the caller refuses on.

commitObservation is deleted rather than narrowed: the raw match was only ever
consulted in this exact case, and in this exact case it is not evidence.

The failure message drops its deadline reference — this refusal never waits.

Refs #1874
Review [P1]: the guard took only the placeholder and the expected text, so it
refused any append whose result happened to equal the placeholder. Value "0" +
`type ".00"` against placeholder "0.00" was refused before a single read, even
though the non-empty pre-dispatch value proves the placeholder is not what is
rendering and a later "0.00" is genuine commit evidence.

The baseline is what decides it, so it is now an input. placeholderCommitEvidence
returns three states rather than a boolean:

  normalRead         expected differs from the placeholder; the placeholder never
                     enters into the observation
  indistinguishable  expected IS the placeholder and the field was empty, so the
                     placeholder was what rendered and no read can resolve it
  rawValueIsEvidence expected IS the placeholder but the field held content, so a
                     raw match is real

Only .indistinguishable refuses, and it still refuses before the wait, since no
read resolves it. .rawValueIsEvidence reaches the observation and settles on the
raw match, which the normalized read would otherwise hide.

commitObservation returns for that third state, now scoped by evidence rather
than applied unconditionally as in the revision that made raw equality a
false success. Both readings stay closures, so normalRead — the ordinary case —
never pays for the raw read.

Refs #1874
@thymikee
thymikee force-pushed the fix/1874-synthesized-commit-timeout branch from afe99bd to 12612b5 Compare August 21, 2026 11:11
@thymikee

Copy link
Copy Markdown
Member Author

P1 addressed in a58410266. The non-empty-baseline/raw-value evidence state is removed: a placeholder-equal final value is now conservatively TEXT_INPUT_COMMIT_NOT_OBSERVED before any post-dispatch poll, because a handler may clear the field and expose the same placeholder rendering. The wait and ordinary text normalization share one placeholder matcher. The clear-after-dispatch case was red on the old model, green after the fix, and red again when the guard was deliberately removed. Final iOS simulator run: 2 focused tests, 0 failures; adjacent policy run: 8 tests, 0 failures. pnpm check:affected --run passed before push, and exact-head CI is running.

@thymikee

Copy link
Copy Markdown
Member Author

Re-reviewed exact head a584102: the clear-after-dispatch P1 is resolved. Placeholder-equal expected values are now conservatively indeterminate before polling, and the new regression proves the old non-empty-baseline/raw-match path would report success over an empty field. The obsolete evidence/raw-read branch was removed and the docs match the conservative contract. Completed checks are green; one iOS Smoke job is still running.

@thymikee thymikee added the ready-for-human Valid work that needs human implementation, judgment, or maintainer merge label Aug 21, 2026
@thymikee

Copy link
Copy Markdown
Member Author

Exact head a584102 is now fully green (30/30) and mergeable/CLEAN. The conservative placeholder-equality fix and its regression remain review-clean. Merge-ready.

@thymikee
thymikee merged commit 766d42e into main Aug 21, 2026
30 checks passed
@thymikee
thymikee deleted the fix/1874-synthesized-commit-timeout branch August 21, 2026 13:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-for-human Valid work that needs human implementation, judgment, or maintainer merge

Projects

None yet

1 participant