Skip to content

Typed operations, engines, workspace, query & MCP (#689) - #690

Open
tony wants to merge 205 commits into
masterfrom
engine-ops
Open

Typed operations, engines, workspace, query & MCP (#689)#690
tony wants to merge 205 commits into
masterfrom
engine-ops

Conversation

@tony

@tony tony commented Jun 21, 2026

Copy link
Copy Markdown
Member

Summary

Adds the typed experimental stack under libtmux.experimental: operations,
engines, eager/lazy/async objects, plans, declarative workspaces, live pane
queries, and an optional MCP server.

The work is additive and remains outside libtmux's stable-versioning policy. It
implements the architecture from #688 and #689 without changing the existing
public ORM API.

What ships

Typed operations and results

  • 58 frozen, keyword-only operation classes across Server, Session, Window,
    Pane, and Client scopes.
  • Constructor signatures expose only target parameters accepted by the
    underlying tmux command.
  • Typed result classes preserve argv, status, output, created identifiers, and
    incomplete composed-operation failures.
  • Lazy plans resolve typed forward references and support sequential,
    semicolon-folded, and {marked} dispatch with identical result semantics.
  • Serialization, MCP schemas, generated API signatures, and workspace rebasing
    share the same constructor-field contract.

Engines and command boundaries

Transport Sync Async
Subprocess SubprocessEngine AsyncSubprocessEngine
In-memory testing MockEngine AsyncMockEngine
Control mode (tmux -C) ControlModeEngine AsyncControlModeEngine
Native imsg ImsgEngine

Every engine returns the same CommandResult shape. Direct argv, control-mode
lines, and imsg frames preserve literal semicolons, newlines, carriage returns,
and leading-dash positional values. Typed separator provenance prevents user
data from becoming a second tmux command.

Objects, queries, and workspaces

  • Eager, lazy, and async Server/Session/Window/Pane/Client objects use the same
    operation spine.
  • panes() provides lazy filtering, ordering, limiting, mapping, and folded
    per-pane commands against either a live engine or snapshots.
  • Declarative Workspace values compile into inspectable plans, support YAML,
    environment and option propagation, host-step fold boundaries, workspace
    sets, and tmux 3.7 floating panes.
  • Create navigation fails with a typed diagnostic when a successful engine
    result omits the identifier required to construct the next object.

MCP server

The optional libtmux[mcp] extra provides curated tools, one tool per
operation, plan tools, workspace tools, resources, prompts, and output
monitoring. Safety is evaluated from the concrete payload across direct,
per-operation, plan, workspace, and event paths. Executable shell,
configuration, environment, option, and active-format payloads require the
destructive tier; target-aware tools also protect the caller's own tmux
hierarchy.

Documentation and developer experience

  • A dedicated operations reference is grouped by Server, Session, Window,
    Pane, and Client, with one generated API page per operation.
  • Querying has separate hierarchy, QueryList, tmux-format, Neo, and snapshot
    guides with API references, examples, happy paths, and failure paths.
  • Every engine has a reference page and relevant tutorial.
  • User examples run against isolated real tmux servers; MockEngine is
    reserved for explicitly offline engine/testing documentation.
  • Generated constructor prose comes from checked NumPy Attributes sections,
    and operation badges/cross-links use the normal gp-sphinx API presentation.
  • Documentation examples execute in pytest/MyST sandboxes and the Sphinx build
    verifies the operation registry cannot drift from its pages.

Verification

  • uv run ruff check . --fix --show-fixes
  • uv run ruff format .
  • uv run ty check
  • uv run mypy
  • uv run py.test --reruns 0 -vvv: 3,104 passed, 1 skipped
  • just build-docs: 141 pages built successfully
  • Live adversarial probes passed across subprocess, async subprocess,
    control-mode, async control-mode, and imsg transports.

Refs #688, #689.

@codecov

codecov Bot commented Jun 21, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 75.82763% with 847 lines in your changes missing coverage. Please review.
✅ Project coverage is 79.96%. Comparing base (9e80112) to head (c249409).
⚠️ Report is 8 commits behind head on master.

Files with missing lines Patch % Lines
scripts/mcp_swap.py 78.79% 175 Missing and 50 partials ⚠️
src/libtmux/experimental/engines/imsg/base.py 54.96% 149 Missing and 28 partials ⚠️
src/libtmux/experimental/engines/control_mode.py 68.55% 94 Missing and 34 partials ⚠️
...libtmux/experimental/engines/async_control_mode.py 75.00% 85 Missing and 41 partials ⚠️
src/libtmux/experimental/engines/imsg/v8.py 76.47% 39 Missing and 17 partials ⚠️
src/libtmux/experimental/mcp/__init__.py 47.61% 28 Missing and 5 partials ⚠️
docs/_ext/tmuxop/render.py 84.86% 15 Missing and 8 partials ⚠️
docs/_ext/tmuxop/domain.py 88.18% 10 Missing and 5 partials ⚠️
src/libtmux/experimental/engines/base.py 88.49% 8 Missing and 5 partials ⚠️
src/libtmux/experimental/mcp/_policy.py 90.65% 5 Missing and 5 partials ⚠️
... and 7 more
Additional details and impacted files
@@             Coverage Diff             @@
##           master     #690       +/-   ##
===========================================
+ Coverage   52.45%   79.96%   +27.51%     
===========================================
  Files          26      234      +208     
  Lines        3729    16260    +12531     
  Branches      747     2057     +1310     
===========================================
+ Hits         1956    13003    +11047     
- Misses       1469     2538     +1069     
- Partials      304      719      +415     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@tony tony changed the title Typed operations and engines: inert op spine (#689) Typed operations and engines: spine + 4 engines + facades (#689) Jun 21, 2026
@tony tony changed the title Typed operations and engines: spine + 4 engines + facades (#689) Typed operations and engines Jun 21, 2026
@tony tony changed the title Typed operations and engines Typed operations & engines: spine, 6 engines, plans, models, facades (#689) Jun 21, 2026
tony added a commit that referenced this pull request Jun 21, 2026
why: Record the experimental operations/engines layer for the
upcoming release so the unreleased section tracks what landed.

what:
- Add a "What's new" deliverable under the unreleased 0.59.x section
  for the experimental operations and engines layer (#690)
- Defer the release lead paragraph until the version is cut
@tony

tony commented Jun 21, 2026

Copy link
Copy Markdown
Member Author

Code review

Found 2 issues:

  1. LazyPlan resolves a forward SlotRef only for target, never for src_target, so a dual-target op (JoinPane, SwapPane, MovePane, BreakPane, SwapWindow, MoveWindow, LinkWindow) whose src_target comes from an earlier plan.add(...) reaches render() with the slot unresolved and raises TypeError: cannot render an unresolved SlotRef. (bug: _resolve() substitutes operation.target but not operation.src_target, even though serialize.py already handles both)

def _resolve(
operation: Operation[t.Any],
bindings: dict[int, str],
) -> Operation[t.Any]:
"""Substitute a :class:`SlotRef` target with a captured concrete id."""
target = operation.target
if not isinstance(target, SlotRef):
return operation
try:
concrete = bindings[target.slot] + target.suffix
except KeyError as error:
msg = (
f"slot {target.slot} has no captured id yet; a plan step can only "
f"target an earlier step that creates an object"
)
raise OperationError(msg) from error
return dataclasses.replace(operation, target=_target_from_id(concrete))

  1. SaveBuffer declares contradictory metadata: safety = "mutating" together with effects = Effects(read_only=True), where read_only is documented as "does not change tmux state". Its read peer ShowBuffer uses safety = "readonly", and a consumer filtering registry.select(lambda s: s.safety == "readonly") would omit save_buffer despite effects.read_only=True. (bug: inconsistent safety/effects declarations)

result_cls = AckResult
safety = "mutating"
effects = Effects(read_only=True)

🤖 Generated with Claude Code

- If this code review was useful, please react with 👍. Otherwise, react with 👎.

@tony

tony commented Jun 21, 2026

Copy link
Copy Markdown
Member Author

Code review

Found 1 issue:

  1. LazyPlan resolves forward SlotRefs on every dispatch path except the {marked} fold's decorates. _drive resolves the create op but builds decorates raw, so a chainable dual-target decorate (SwapPane/JoinPane/MovePane) whose src_target is a forward slot reaches render_marked unresolved and raises TypeError: cannot render an unresolved SlotRef. The single-op and chain paths both call _resolve; this one does not. (bug: decorates = [self._operations[i] for i in decorate_idx] skips _resolve, so src_target SlotRefs survive into render under MarkedPlanner)

create_idx, *decorate_idx = step.indices
create = _resolve(self._operations[create_idx], bindings)
decorates = [self._operations[i] for i in decorate_idx]
merged: CommandResult = yield _Chain(
render_marked(create, decorates, version),
)

🤖 Generated with Claude Code

- If this code review was useful, please react with 👍. Otherwise, react with 👎.

@tony

tony commented Jun 22, 2026

Copy link
Copy Markdown
Member Author

Code review

No issues found. Checked for bugs and CLAUDE.md compliance.

🤖 Generated with Claude Code

1 similar comment
@tony

tony commented Jun 22, 2026

Copy link
Copy Markdown
Member Author

Code review

No issues found. Checked for bugs and CLAUDE.md compliance.

🤖 Generated with Claude Code

tony added a commit that referenced this pull request Jun 27, 2026
why: Record the experimental operations/engines layer for the
upcoming release so the unreleased section tracks what landed.

what:
- Add a "What's new" deliverable under the unreleased 0.59.x section
  for the experimental operations and engines layer (#690)
- Defer the release lead paragraph until the version is cut
tony added a commit that referenced this pull request Jun 28, 2026
why: Record the experimental operations/engines layer for the
upcoming release so the unreleased section tracks what landed.

what:
- Add a "What's new" deliverable under the unreleased 0.59.x section
  for the experimental operations and engines layer (#690)
- Defer the release lead paragraph until the version is cut
tony added a commit that referenced this pull request Jul 4, 2026
why: Record the experimental operations/engines layer for the
upcoming release so the unreleased section tracks what landed.

what:
- Add a "What's new" deliverable under the unreleased 0.59.x section
  for the experimental operations and engines layer (#690)
- Defer the release lead paragraph until the version is cut
tony added a commit that referenced this pull request Jul 4, 2026
why: Record the experimental operations/engines layer for the
upcoming release so the unreleased section tracks what landed.

what:
- Add a "What's new" deliverable under the unreleased 0.59.x section
  for the experimental operations and engines layer (#690)
- Defer the release lead paragraph until the version is cut
@tony tony changed the title Typed operations & engines: spine, 6 engines, plans, models, facades (#689) Typed operations, engines, workspace, query & MCP (#689) Jul 4, 2026
@tony

tony commented Jul 5, 2026

Copy link
Copy Markdown
Member Author

Code review

No issues found. Fresh review of the current HEAD (4393b02a) — the ops / engines / facade / workspace / query / MCP surface was checked for bugs and AGENTS.md compliance and is clean. (The prior review comments predate ~12 days of commits.)

🤖 Generated with Claude Code

tony added a commit that referenced this pull request Jul 5, 2026
why: Record the experimental operations/engines layer for the
upcoming release so the unreleased section tracks what landed.

what:
- Add a "What's new" deliverable under the unreleased 0.59.x section
  for the experimental operations and engines layer (#690)
- Defer the release lead paragraph until the version is cut
tony added a commit that referenced this pull request Jul 11, 2026
why: Record the experimental operations/engines layer for the
upcoming release so the unreleased section tracks what landed.

what:
- Add a "What's new" deliverable under the unreleased 0.59.x section
  for the experimental operations and engines layer (#690)
- Defer the release lead paragraph until the version is cut
tony added a commit that referenced this pull request Jul 12, 2026
why: Record the experimental operations/engines layer for the
upcoming release so the unreleased section tracks what landed.

what:
- Add a "What's new" deliverable under the unreleased 0.59.x section
  for the experimental operations and engines layer (#690)
- Defer the release lead paragraph until the version is cut
tony added 18 commits July 26, 2026 18:49
why: Activate gp-sphinx card styling and responsive layout rules for
custom operation entries.

what:
- Wrap operation entries in the shared API card shell
- Assert the rendered shell contract in the Sphinx integration test
why: Let users choose a query API by freshness, filtering boundary,
and failure behavior instead of navigating one mixed topic.

what:
- Add hierarchy, QueryList, tmux-format, Neo, and snapshot guides
- Test independent happy and sad examples plus styled API sections
- Preserve traversal and filtering URLs with redirects
why: Keep the verified documentation extension and tests in the repository canonical format.

what:
- Apply ruff line wrapping to the operation domain and query tests
why: Operation cards flattened Python signatures into literal text, bypassing Sphinx Python-domain semantics and gp-sphinx API styling.

what:
- Render operation classes and constructors as native Python descriptions
- Register public class and constructor inventory targets
- Test RST and MyST rendering against semantic gp-sphinx markup
why: Wall-clock corrections can move backward and invalidate timeout
measurements in retry_until and its tests.

what:
- Measure retry deadlines with the monotonic clock
- Measure elapsed-time assertions with the same clock
why: Operation pages need real tmux proofs and API references that
match the normal gp-sphinx presentation.

what:
- Replace mock operation examples with live typed outcomes
- Add engine, tutorial, and result API references
- Align operation cards, source links, parameters, and badges
- Test documentation inventory, rendering, and tmux behavior
why: CI checks the documentation extension under a different package
root and against Sphinx's installed domain annotations.

what:
- Use package-relative imports for both Sphinx and mypy discovery
- Follow inherited Sphinx domain attribute typing
- Inspect public constructor signatures without unsafe __init__ access
why: tmux releases disagree on whether list-clients includes a
suspended client, making the documented output version-dependent.

what:
- Verify the real AckResult and empty command output
- Preserve the portable session-survival assertion
- Document client-list visibility as version-dependent
why: Deleting a control-owned session can race tmux's deferred global
notifications and terminate supported tmux servers.

what:
- Attach control clients to safe existing sessions without updating the
  environment
- Bootstrap empty or unsafe servers through tracked subprocess execution
- Serialize lifecycle transitions without blocking dependent fallback
  commands
- Cover live hook, concurrency, cancellation, and cross-version paths
why: CI type-checks against Python 3.10, where Task.cancelling is not
available in the asyncio stubs.

what:
- Replace the version-specific cancellation probe with a portable task wait
- Make the shared startup future's nullable type explicit
- Align lifecycle test overrides and terminal paths with their contracts
why: Whole-file backups could be overwritten or unwound out of order,
and malformed state could abort configuration recovery.

what:
- Preserve first backups and enforce per-config LIFO ordering
- Checkpoint state before config writes and each successful revert
- Fail closed on missing backups, corrupt state, and malformed configs
- Cover repeat swaps, partial failures, and recovery diagnostics
why: Experimental operations, engines, and agent tools could diverge from
their documented constructor, result, and command-boundary contracts.

what:
- Make Attributes the checked source for generated constructor API prose
- Expose only tmux-supported targets across Python, MCP, and payloads
- Preserve create results and tmux 3.7 composed-operation invariants
- Gate executable MCP payloads across every executable tool path
- Preserve literal arguments across every experimental tmux transport
- Add the direct YAML dependency and tested failure guidance
why: Show how typed Python chains become folded tmux command sequences
while one asynchronous control-mode client carries the work.

what:
- Add a tabbed live tutorial for forward references and planner folding
- Replace simulated plan examples with real tmux execution
- Test visible compiled commands against actual control-mode dispatches
- Add API destinations and navigation for plan concepts
why: First-party record fields must render with complete semantic
prose, while gp-sphinx a36 now owns documented-member deduplication.

what:
- Expand the runtime contract across supported record declarations
- Define inheritance, ordering, and exemption rules in AGENTS.md
- Remove the obsolete no-undoc-members workaround and source test
why: CI must type-check the untyped doctest dependency boundary and
the heterogeneous results produced by a compiled operation plan.

what:
- Describe the consumed doctest finder interface with a protocol
- Narrow the terminal plan result before reading message text
why: Operation's ten class variables were documented as a hand-written
definition list under Notes, because an Attributes entry for a class
variable used to be dropped from the build. gp-sphinx renders one now,
with the annotation and value alongside the prose, so the workaround
costs a reader the type and default it cannot state.

what:
- Move kind, command, scope, result_cls, chainable, primitive, safety,
  effects, min_version, and flag_version_map into the Attributes
  section, dropping the Notes list

Requires a gp-sphinx release carrying the class-variable rendering;
under the pinned 0.0.1a36 these descriptions do not render.
why: Engine users need transport-specific examples that expose real output
and boundaries without treating mock results as server evidence.

what:
- Add one executable first-success example per concrete engine
- Map every engine to a focused live or offline tutorial
- Test tutorial ownership and transport boundaries
why: gp-sphinx removes tabs.js after rendering, leaving pages with a missing
asset reference even though inline tabs operate through CSS.

what:
- Filter only tabs.js from Sphinx page contexts at late priority
- Verify final tab markup and assets with a one-page Sphinx build
tony added 11 commits July 26, 2026 19:01
why: ruff's default rule set, adopted on master, enforces PLR0402.
`import a.b as b` and `from a import b` bind the same name, and the
from-form is the one the rest of the suite uses.

what:
- Rewrite the engines and ops submodule imports in the docs tests

https://docs.astral.sh/ruff/rules/manual-from-import/
why: ruff's default rule set enforces FURB188. The conditional slice and
`str.removeprefix` are equivalent for a single-character prefix, and the
method says what the code is doing.

what:
- Replace the guarded slice in `ControlNotification.parse`

https://docs.astral.sh/ruff/rules/slice-to-remove-prefix-or-suffix/
why: ruff's default rule set enforces PYI025. Bare `Set` reads as the
`set` builtin at the use site, but it is the abstract collection; the
annotation accepts any set-like, not just `set`.

what:
- Import `collections.abc.Set` as `AbstractSet` and use it in
  `get_objects`' docnames annotation

https://docs.astral.sh/ruff/rules/unaliased-collections-abc-set-import/
why: ruff's default rule set enforces FLY002, which wants an f-string in
place of a static join. An f-string is the wrong shape here: tmux format
specifiers are `#{...}`, so every brace would need doubling, and the
eight fields would collapse onto one line. Binding the tuple keeps one
field per line and leaves the join non-static.

what:
- Extract the tmux format specifiers into `_DONE_FIELDS` and join that

`_DONE_FORMAT` is unchanged.

https://docs.astral.sh/ruff/rules/static-join-to-f-string/
why: ruff's default rule set enforces ISC004. Implicit concatenation
inside a list literal reads like a missing comma between elements, which
is how a segment silently merges into its neighbour. Explicit parens make
each element's extent unambiguous.

what:
- Wrap the four multi-line instruction segments in parentheses

The rendered instructions are byte-identical, with and without events.

https://docs.astral.sh/ruff/rules/implicit-string-concatenation-in-collection-literal/
why: ruff's default rule set enforces PYI034. Annotating `__enter__`,
`__aenter__`, and `__new__` with the concrete class loses the subclass:
`with SubEngine() as e` inferred the base, so subclass-only attributes
read as errors and the wrong type propagated to callers.

what:
- Return `Self` from `ControlModeEngine.__enter__`,
  `AsyncControlModeEngine.__aenter__`, and `CommandSeparator.__new__`
- Same for the forged separator in the engine base tests
- Import `Self` from `typing_extensions` under `TYPE_CHECKING`, matching
  the rest of the package's 3.10 backport pattern

https://docs.astral.sh/ruff/rules/non-self-return-type/
why: ruff's default rule set enforces FURB192. Sorting a whole name set
to read its first element states the intent less directly than asking for
the minimum.

what:
- Use `min()` for the two suggested-server picks in the doctor output

Ruff marks the fix unsafe because `sorted(x)[0]` raises `IndexError` on an
empty sequence where `min(x)` raises `ValueError`. Both call sites sit
behind an emptiness guard on the line above, so neither can be reached
with an empty set.

https://docs.astral.sh/ruff/rules/sorted-min-max/
why: ruff's default rule set enables BLE001, which fires at nine sites
where catching everything is the contract rather than a mistake. Each
handler records the failure for a caller instead of swallowing it, so
narrowing the except clause would lose the failure mode it exists to
report.

what:
- Scope per-file ignores, with the reason, to the async control-mode
  supervisor, the MCP event drainer, the error-result middleware, the
  schema fallbacks, and the safety-gate tier assertions

The supervisor and reader tasks re-raise `CancelledError` before the
catch-all, so cancellation still propagates.

https://docs.astral.sh/ruff/rules/blind-except/
why: ruff's EXE001 fires on CI but never locally: it short-circuits on
WSL, where every file reports executable, so a Linux runner sees a
shebang on a mode-644 file that this machine cannot. Both scripts also
declared PEP 723 dependencies their `python3` shebang could not satisfy,
so `./scripts/mcp_swap.py` died on a missing import.

what:
- Set mode 100755 on `scripts/bench_engines.py` and `scripts/mcp_swap.py`
- Point both shebangs at `uv run --script`, which resolves the inline
  dependency block

`uv run scripts/<name>.py`, the form the docs use, is unaffected.

https://docs.astral.sh/ruff/rules/shebang-not-executable/
why: `use-local` could only point the agents at this working copy, so
reviewing a branch meant checking it out first. The writer was also
careless with files it does not own: it replaced a symlinked config with
a regular file, dropped its permission bits, re-escaped non-ASCII text it
never read, appended a newline Claude never wrote, and recorded
`config_path` as recovery identity — so a link repointed after a swap
sent `revert` into someone else's file.

what:
- Add `use-local --pr N`, writing `uvx --from git+<remote>@refs/pull/N/head
  <entry>`; the ref lives on the base repo so fork PRs work unchanged and
  nothing is checked out for `revert` to clean up
- Probe that spec with one MCP `initialize` round trip before any config
  is touched, so a bad ref fails once here instead of inside every agent;
  `--no-preflight` skips it
- Resolve symlinks and carry the target's mode through `atomic_write`,
  and record the resolved `SwapEntry.target_path` for `revert` to use
- Take the file's own trailing-newline convention in `dump_config_bytes`
  and stop escaping characters outside the entry being swapped
- Compare argv exactly in `_points_at`, so `--entry` is not swallowed by
  the already-local short-circuit, and label a PR before the pin branch
- Cover pull-request targeting, JSON writer fidelity and symlinked
  configs, and guard that `fake_home` lists every registered CLI

The `libtmux-engine-mcp-dev` state slug, the entry-derived server name,
and the recovery-stack guards this repo added stay as they were.
why: Two more agent CLIs are installed here and neither could be swapped.
Both keep their MCP config in JSONC, which the JSON writer would have
reserialized -- stripping every comment out of a file the user wrote by
hand. opencode also disagrees with the other six about how one entry is
spelled: its container key is `mcp`, argv goes into a single `command`
array, and the environment table is called `environment` (an `env` key is
dropped in silence, and a scalar `command` is a decode error that stops
opencode starting at all).

what:
- Add a JSONC codec that edits by text splice, so comments, trailing
  commas, indent width and a missing final newline all survive a swap;
  values still come from stdlib `json`, so escape semantics are the
  standard library's
- Move the per-CLI branching onto `CLIInfo.container` and
  `CLIInfo.dialect`, so `get_server`, `set_server`, `delete_server` and
  `_all_server_specs` no longer carry a chain of CLI-name tests
- Register `opencode` ($XDG_CONFIG_HOME/opencode/opencode.jsonc) and `pi`
  (~/.pi/agent/mcp.json), normalising opencode's array command back to the
  portable scalar-plus-args spec so the already-local check still fires
- Ignore a relative XDG_CONFIG_HOME, which the spec requires and which
  otherwise made the recorded backup path depend on the working directory
- Say in `detect` that pi has no MCP client of its own: the file is read
  by the third-party pi-mcp-adapter, so without it the swap does nothing
- Derive the detect column width from the longest CLI name
- Cover both CLIs end to end plus JSONC fidelity, the one-delimiter member
  removal, key escaping, and comment survival on insert
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant