Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@
- `diff --application` reads bounded local factories returning OpenAI Agents SDK and Google ADK tool lists. It keeps the returned members and their construction locations, and reports unread constructor identity, import effects, escaping values, or unresolved callers as coverage gaps rather than treating them as established wiring. This is static source analysis; it does not execute a factory or establish runtime enforcement. (#874, #964)
- Constructor ownership still withholds canonical SDK Runner execution consumers and anonymous callable or generator carriers. These are static coverage limits; readable candidate tools and handoffs remain visible, and incomplete scans do not report them as removed. The direct `from . import agent` ADK package layout is supported. When a binding graph is incomplete, a capability comparison now withholds only what depends on absence (removed and narrowed capabilities, resolved findings) and still reports added, changed and unknown-direction changes, saying so in its notes; a remote Google ADK `McpToolset` whose tools cannot be listed is not itself incomplete binding evidence, so a changed endpoint, `os.environ[...]` credential reference or tool filter on it, nested stdio parameters included, is reported again. An ADK sub-agent behind a constructor gap stays a candidate with an incomplete edge instead of being reported as unmatched. A module that re-exports `eval` or `exec` from `builtins` is no longer treated as an unused import while another module imports it. The proof that no repository file provides a standard-library module reads the directory listing, so it is the same on case-sensitive and case-insensitive filesystems. (#964)
- **An MCP row names the `env_vars` entries that changed.** Replaying openai/codex-security#1281, a `.mcp.json` that added `CODEX_SECURITY_PLUGIN_ROOT` to a server's `env_vars` array and changed nothing else was a `changed` row reading `no difference in the command name launch_codex_security_mcp, launch arguments, env key names or header key names; the change is in a detail this output does not show, such as the command's path or another setting`, which pointed at the command's path for a change to a variable name. An MCP server grant now publishes `env_var_names` (host-grants `0.9`, extended in place): the plain names its `env_vars` list declares, in declared order, never a `NAME=value` entry, an object, a token-shaped string or an entry the digest's redaction rewrites. The row reads `codex-security: env_vars names +CODEX_SECURITY_PLUGIN_ROOT` on `diff` (text and `--json`), `verify`, the PR comment and `check`, beside `env keys` and `header keys` (`env keys +API_BASE -DEBUG; env_vars names +X -Y`); the same names in another order read `env_vars names in a different order`, and an added or removed server's cell lists them. The label is the key the list is declared under, so no claim is made that a variable is set or read. No value is published or printed, and rotating an `env` value is still no row, because `config_sha256`'s input redacts every `env` value. The field is display-only, left out of grant equality, the inventory digests and saved baselines, so no row, direction, `expands`, severity, expansion signal or `check` decision moves; a change confined to an unpublished entry still says it is not shown. See [STABILITY.md](STABILITY.md#mcp-env-var-names-795). (#795)
- **The five MCP tools describe themselves in `tools/list`.** Each `shipgate.*` tool was listed with an empty description; each now states its task, its inputs and that it runs no git, Git-backed host comparison, scan or verify, with names, input schemas and annotations unchanged. (#800)
- **An OpenShell policy row names the declarations that changed.** Replaying openshift-kni/ai-sandbox#29, a selected policy that removed `/sandbox/.uv/python` from `filesystem_policy.read_only`, changed binary selectors and added GitHub release destinations printed `authored, OpenShell 0.1.2, 6 endpoint(s), facts <digest> → …7 endpoint(s), facts <digest>`, which names none of them. `diff` (text and `--json`), `verify`, `check` and the PR comment now print the difference: `filesystem read_only -/sandbox/.uv/python`, `destination aiplatform.googleapis.com:443 (l4) binaries +/tmp/**/node_modules/…/claude -/usr/local/bin/claude`, `removed destination oauth2.googleapis.com:443 (l4) via /usr/local/bin/claude` and `added destinations objects.githubusercontent.com:443, release-assets.githubusercontent.com:443 (l4) via …`, with changed request declarations, `include_workdir`, Landlock compatibility and process identity named the same way. Each list names its first entries and counts the rest. Only values the OpenShell inventory already publishes are named, and a value that would change under redaction prints as `<redacted>`. Query and parameter constraints and endpoint options appear by field name, not value. A policy selected or removed whole lists what it declares and which sections and inspected endpoints took an upstream default, as a description and not a proof that authority grew. Display only: the published `rows[]`, direction, `expands`, severity, expansion signals and `check` decisions are unchanged, the existing unknown-direction and runtime-limit wording stays in each row's reason, and no schema changes. (#968)
- **`diff` classifies a large base tree about eight times faster.** Translating each boundary glob into a regular expression was repeated for every path in the tree and was 74–86% of first-run `diff` time on two large repositories (#698's measurement). The compiled pattern is now cached per pattern; which paths match is unchanged, and a test checks the cached matcher against the previous implementation on over 10,000 random pattern and path pairs. (#698)
- **`diff --application` leads with one finding per changed agent.** The text opens with a `Findings:` line and, for each agent that has a row, a finding whose first line stands alone: the tools the change added (`+`), removed (`-`) or changed (`~`), or only candidates (`?`); what they newly reach (the outbound call or library effect, its host, the credentials it sends by name, the parameters the model controls) or, for a changed tool, what it still reaches; and where reading stopped (`not read beyond memory_bank.py:531`), with the first unread hop and a count of the rest per tool. A finding with a binding or a reach it did not read is `partial`, though `comparison_status` still speaks of bindings only. A question specific to the change follows ("should `agent` reach `POST …` through `act`, sending `env WEATHER_TOKEN`, with the model controlling `city`?"; the same call newly taking a model-controlled argument; a removed tool; a changed one and what changed). Rows that are `not_established` for the same reason are one line with a count and one example, however many tools and agents share it, and limits no row states are one line per agent; effect evidence is counted once instead of `write (provisional: unknown effect)` per row. Per-row text follows under `Detail:` for up to 10 rows; above that the text says how many it did not print and `--json` carries every row, gap and reach limit. The JSON gains a top-level `summary` with the same content (`counts`, `findings`, `effects`, `causes`, `not_read`), each entry naming the rows it summarizes by index; it states nothing a row does not, and no row, status, direction, gap, scope or exit code changes. `application_comparison_schema_version` stays `0.4`, unreleased, with `summary` added. `comparison_id` digests it with the rows; `benchmark/application-q2/summarize.py` leaves it out of `answer_id`, so a hand score keeps judging the answer it read. On the 49-PR development corpus (`benchmark/application-q2/`) against `main` at `e703b41b`, `summarize.py` counts the same statuses and rows and every answer has the same `answer_id`. VidulaWickramasinghe/O.R.I.O.N#126's text goes from 11,926 lines to 24, jpka/attest#3's findings fit in 7 lines before its rows and Tiendat2703/MIS_TALENT#7's in 10; the committed goldens in `benchmark/application-q2/goldens/` hold six real answers. (#914)
Expand Down
2 changes: 1 addition & 1 deletion docs/distribution-surfaces.md
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,7 @@ and this document are checked against each other by
| `examples` | `examples/` | `executable_pin`, `merge_verdict_vocabulary` | `test_executable_pin_resolves_in_a_published_channel`, `test_surface_enumerations_match_the_engine_vocabulary`, `tests/test_adopter_pins_resolve.py::test_every_pin_init_writes_into_an_adopter_repo_names_the_published_release` | Illustrative CI wiring. It gates on the engine's merge verdict rather than restating the release-decision set, so only the verdict claim is registered. |
| `policies` | `policies/` | — | — | Manifest fragments only. Answers no question the CLI answers: they are *inputs* the engine evaluates, not restatements of its output. |
| `harness` | `harness/` | `merge_verdict_vocabulary`, `release_decision_vocabulary` | `test_harness_holds_no_drifted_copy_of_the_engine_vocabularies` | Internal adoption-measurement harness, not an adopter-facing surface. It grades artifacts from whichever build a cell ran, so it carries literal vocabularies — each one derived from the engine's and checked against it. |
| `mcp_server` | `src/agents_shipgate/mcp_server/` | — | — | Transport only. Every answer it returns is produced by calling the CLI in-process, so it has nothing of its own to drift. |
| `mcp_server` | `src/agents_shipgate/mcp_server/` | — | — | Transport only. Every answer it returns is produced by calling the CLI in-process. Its only words of its own are each tool's `tools/list` description, which enumerates no verdict set and names no pin or contract floor, and reads the schema IDs it names from the engine's constants, so it adds no claim; `tests/test_mcp_server.py` holds each description to its tool's inputs and boundary, in-process and over stdio, and CI's `mcp extra` job checks the installed command sends them (#800). No tool runs git, the Git-backed host comparison (`diff --base`), `scan` or `verify`. |
| `design_partner_runbook` | `docs/design-partner-verifier-pilot.md` | `contract_floor`, `executable_pin`, `placeholder_ownership` | `test_executable_pin_resolves_in_a_published_channel`, `test_runbook_channel_table_states_the_released_contract_correctly`, `test_surface_routes_human_owned_placeholders_to_a_human`, `tests/test_adopter_pins_resolve.py::test_every_pin_init_writes_into_an_adopter_repo_names_the_published_release` | Version-specific by construction: it names one channel per partner and states what the released build does *not* emit. |
| `human_entry_path` | `README.md`, `docs/quickstart.md` | `contract_floor`, `executable_pin`, `merge_verdict_vocabulary`, `placeholder_ownership`, `release_decision_vocabulary`, `report_schema_pin` | `test_executable_pin_resolves_in_a_published_channel`, `test_the_human_entry_path_states_what_the_published_build_provides`, `test_surface_enumerations_match_the_engine_vocabulary`, `test_surface_routes_human_owned_placeholders_to_a_human`, `test_surface_names_the_current_report_schema` | States its vocabularies as Markdown reference tables rather than braced sets; the vocabulary reader was taught that shape by [#498](https://github.com/ThreeMoonsLab/agents-shipgate/issues/498) rather than exempted from it. The quoted `diff` answers, the auto-detected `origin/HEAD` base, the documented missing-base recovery and the channel statements are held to this tree's output and to `.github/release-channels.json` by `tests/test_host_diff_entry_docs.py` (#779), which the claims vocabulary has no row for. The same module holds the build the pages say their quotes came from to a record of the answers the newest published release prints, taken from that build at release-runbook step 8 (#778). The quickstart's detector continue rule names `suggested_sources`, `codex_plugin_candidates` and `host_boundary_candidates`, so a host-only repository is not stopped where `detect` routes it forward; `tests/test_public_surface_contract.py::test_detect_continue_rules_do_not_stop_a_host_only_repository` holds it and `llms.txt`'s rule to those fields (#792). |
| `agent_instructions` | `AGENTS.md`, `docs/agent-recipes.md`, `docs/agents/`, `docs/target-repo-agent-snippets.md` | `executable_pin`, `placeholder_ownership`, `report_schema_pin` | `test_executable_pin_resolves_in_a_published_channel`, `test_surface_routes_human_owned_placeholders_to_a_human`, `test_surface_names_the_current_report_schema` | States no contract floor of its own — it links [`agent-contract-current.md`](agent-contract-current.md) for the versions. What it does answer is who may fill a manifest placeholder, on the copy a coding agent actually reads, and it carries an Action ref, a `shipgate_version:` input and an install floor a reader runs. The `AGENTS.md` host-configuration review bullets name the manifest-free `diff` route and quote none of its output, so they add no claim; the quoted `diff` answers belong to the `human_entry_path` row (#792). |
Expand Down
16 changes: 13 additions & 3 deletions docs/mcp-server.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,15 @@ Claude Code registration (`.mcp.json`):
| `shipgate.preflight` | `{workspace?, config?, plan?, changed_files?, diff_text?, capability_request?, base_preflight?}` | exact `PreflightResultV6` (preflight `0.6`) |
| `shipgate.explain` | `{check_id}` or `{fingerprint, report_path}` | deterministic check/finding explanation JSON |
| `shipgate.capabilities` | `{config}` or `{base_lock, head_lock}` | capability lock or capability lock diff JSON |
| `shipgate.handoff` | `{verifier_path, report_path?, verify_run_path?}` | exact `shipgate.agent_handoff/v8` |
| `shipgate.handoff` | `{verifier_path, report_path?, verify_run_path?}` | exact `shipgate.agent_handoff/v9` |

Each tool's `tools/list` entry carries a description that a connected agent
reads when it picks a tool: the task first, then every input the tool accepts,
what it returns and what it cannot do. No tool runs git or the CLI's Git-backed
host comparison: `shipgate.check` takes diff text the caller already has and
obtains no base or head ref, and `shipgate.capabilities` compares two lock
files that already exist. For a comparison against a base ref, run
`agents-shipgate diff --base <ref>` from the CLI.

`shipgate.check` is the same protocol surface documented in
[`agents/protocol.md`](agents/protocol.md). `shipgate.preflight` is proactive
Expand Down Expand Up @@ -64,8 +72,10 @@ following a verification route or reporting completion.

`shipgate.handoff` is a read-only projection over existing verifier artifacts.
It never runs `verify`, shells out to git, or writes `agent-handoff.json`; it
returns the same `shipgate.agent_handoff/v8` shape that `verify` writes for
agents that need a compact control/release-readiness object.
returns the same `shipgate.agent_handoff/v9` shape that `verify` writes for
agents that need a compact control/release-readiness object. It does not check
that those artifacts still match the working tree: they describe only the tree
that `verify` run checked, so re-run `verify` after any change.

## Trust model

Expand Down
12 changes: 11 additions & 1 deletion scripts/mcp_extra_requirement.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,8 @@

`--stdio-smoke` starts the installed `agents-shipgate mcp-serve` as a real stdio
MCP server, lists its tools through the SDK client, and exits non-zero unless
exactly the five read-only, closed-world tools answer.
exactly the five read-only, closed-world tools answer, each with its own
non-empty description.
"""

from __future__ import annotations
Expand Down Expand Up @@ -69,7 +70,16 @@ def stdio_smoke() -> None:
or annotations.open_world_hint is not False
):
raise SystemExit(f"{tool.name} is not advertised read-only and closed-world")
# #800: a bare name tells a connected agent neither when to call the
# tool nor what it needs. The wording itself is held by
# tests/test_mcp_server.py; this proves the installed command sends it.
if not (tool.description or "").strip():
raise SystemExit(f"{tool.name} is listed with an empty description")
if len({tool.description for tool in tools}) != len(tools):
raise SystemExit("mcp-serve listed two tools with the same description")
print(f"mcp-serve answered over stdio with {len(tools)} read-only tools")
for tool in sorted(tools, key=lambda item: item.name):
print(f"{tool.name}: {tool.description}")


def main(argv: list[str]) -> None:
Expand Down
Loading
Loading