diff --git a/CHANGELOG.md b/CHANGELOG.md index c989180c..428b9492 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -45,6 +45,7 @@ - **OpenShell link retargets retain supported authority expansions.** A selected policy link changing targets still requires reference review, and the same change now also reports every widening established from the captured policies. This covers a linked document moving from enforced request restrictions to audit-only, and an independent policy widening beside a same-byte link retarget. Static comparison and runtime-enforcement limits are unchanged. (#942) - Configured committed verification reads selected OpenShell file-link chains from the same verified Git object store as its application snapshot, including mixed-case registrations and chains of up to eight hops. Generic archives still reject links; configured snapshots still reject unrelated, escaping, cyclic, directory and over-bound links, selected links that would not open as written (through a missing or non-directory component, or with an empty one such as a trailing slash), and submodules. `verification prepare` now captures every host dependency the boundary snapshot reads before building its plan, as configured `verify` already did. Worker replay therefore rejects a selected-link retarget even when policy bytes are identical, and portable plans now bind Claude hook scripts too, so a hook script that is a symbolic link makes the worker reject the plan. Scoped host-comparison archives now resolve in-tree link chains of up to eight hops (was seven). No schema or contract version changes. (#942, #945) - **An empty preflight plan no longer grants merge or completion.** A plan that named no changed file, capability request or host permission request returned the shared `complete` state, whose permissions include `merge` and `report_complete`, with no verifier run behind it. Preflight `0.6` answers it with the new `planning_only` state, which owes no action and authorizes nothing, and every preflight route now denies every permission in both the model and the published schema; `complete` and `review_publishable` cannot appear. Docs-only plans still route to `verify`, and protected surfaces and drift still stop for a human. `0.5` stays frozen and readable as a base preflight. Runtime contract 41 → 42; `minimum_control_contract_version` stays 21. Hooks written by `install-hooks` before this change accept only preflight `0.5`, so their instruction-structure check fails closed until `install-hooks --write` is re-run. See the `planning-only preflight` migration note in `STABILITY.md`. (#610) +- **Published schema documents keep their bytes across releases.** A version bump no longer rewrites agent-handoff v9, agent-result v3 or agent-boundary-result v3, and the generator refuses to change any `docs/*-schema*.json` a release tag carries. (#1044) - Commit the application review benchmark (`benchmark/application-q2/`): the 49 pinned development pull requests, the runner, the hand-scoring protocol, the 2026-09-30 ledger, reproduced from fresh clones with every status, row and comparison id identical (Q2 1/49; re-read, 12 members are not SDK/ADK wiring changes, so Q0 is 37/49, and MIS_TALENT#6 is Q1, so Q1 is 2/49), and a holdout selection rule, the definitions it uses and a recorded repository pool, frozen by digest before the reader changes they will judge. Each release now records `Q2: n/49 development, m/≥30 holdout`. Development evidence only; no success-rate claim. (#908) - **`diff --application` reads tools lists built by an expression.** An OpenAI Agents SDK or Google ADK agent's `tools=` (and an SDK agent's `handoffs=`, an ADK agent's `sub_agents=`) built by a spread, `+`, a conditional, `or`, a filter or another module's list is read member by member instead of reported as one dynamic expression. A part it cannot read — a call, a builder's parameter, `self.tools` — is named with its location on that agent, which stays incomplete, and the members read are still compared. A member held only under a condition carries `bound_when`, and a change to the condition alone is a `changed` row whose direction is not established. `application_comparison_schema_version` 0.2 → 0.3. The Google ADK dynamic-tools limit now names its file and line and its agent, a starred element the ADK reader cannot read no longer leaves the agent's list read as complete, and an imported `handoffs=` list no longer reads as one handoff named after the list. The SDK recovery reason `sdk_literal_tool_list_concatenation_unsupported` is retired: that form is read (#584). For `scan`, an ADK list that came through a name or from another module is not counted as a proven surface, so no `scan` decision moves. On the 49-PR development corpus (`benchmark/application-q2/`), pull requests with any row go from 9 to 11 and those still carrying a tools-list limit from 28 to 24, most of the rest a builder's parameter (#874); MIS_TALENT#7 now reports the `Finance_Agent` row the 2026-09-30 ledger found missing. On 131 open SDK/ADK pull requests, every status is unchanged and two gain rows, all 16 checked against source. (#909) - **`diff --application` reads tools bound as objects.** An MCP server or toolset (Google ADK `McpToolset`; OpenAI Agents SDK `MCPServerStdio`, `MCPServerSse`, `MCPServerStreamableHttp` in `mcp_servers=`), an agent exposed as a tool (SDK `agent.as_tool(...)`, ADK `AgentTool(agent=...)`, and a repository helper that returns one from its single `return`), a hosted or built-in tool (SDK `WebSearchTool`, `FileSearchTool`, `CodeInterpreterTool`, `ComputerTool`; ADK `google_search`, `built_in_code_execution`, `load_memory`) and the SDK's `function_tool(f)` are bindings of their agent, where they used to read as an unresolved tool or "assigned a value, not a function definition". A row's side carries `object`: an MCP server's transport, the URL's host or the program's file name, the names of the credentials it sends and its tool filter; the agent an agent tool wraps; a built-in's name. A changed host, command, transport, credential name, filter or wrapped agent is a `changed` row. A URL's path or query, a header value and a command's arguments are digested (`endpoint_sha256`), never printed. Identity is the import, never the spelling: a same-named local function, a shadowed, reassigned or patched import, a vendored `google/adk` package and another package's `load_memory` are not built-ins. A helper whose return cannot be resolved, an `.as_tool` on a value not identified as an agent, an object changed after it is built, and an `mcp_servers=` member that is not an MCP server stay named limits; a part of an object's identity that is not read (a host from a builder's parameter) leaves the binding present but makes a change to it `not_established`. An SDK agent's `mcp_servers=`, which the comparison did not read before, is read member by member. Read for `diff --application` only: `scan` reads what it did. `application_comparison_schema_version` 0.3 → 0.4. On the 49-PR development corpus (`benchmark/application-q2/`) against `main` at `7c160cfb`, every status is unchanged, pull requests with any row go from 9 to 10, and no row is dropped: cuppibla/adk_tutorial#9's `update_plan` is now an established `added` row (its base `google_search` was unresolved), buriro-ezekia/cinescout-ai#9 gains a `not_established` row naming the base MCP toolset's host, transport and `Authorization` source, and 24 (adk_tutorial#7) and 41 (Lling0000/OpenCMO#41) limits are gone. zendah21/capstone_project#3 is `compared` with `load_memory`, `inspect_schema` and `execute_sql` added. On 131 open SDK/ADK pull requests every status is unchanged; 7 gain rows (14 new rows, 2 of them `not_established`, and 3 candidate removals now established, all checked against source), and 2 gain a named `mcp_servers` limit where a server list was not read before, as 2 development members do. What remains a limit: a helper with more than one `return`, an agent tool whose agent has no literal name, a re-exported built-in, a pydantic settings class, and #874's caller-supplied lists. (#910, #866) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index d75b100f..7040ed7f 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -338,6 +338,12 @@ CI runs `python scripts/generate_schemas.py --check` and fails fast with a unified diff if a committed schema drifts from the live model. The same drift is also caught by `tests/test_schema_roundtrip.py`, so your test suite will reject the change locally before CI does. +A `docs/*-schema*.json` file that a release tag carries is published and +keeps its bytes: it is pinned in `scripts/published_schema_pins.json`, and +the generator writes nothing and exits 1 rather than change one. Put the +change in a successor schema version instead (#1044). The schemas are +rendered by the locked pydantic (`constraints/dev.txt`); another version can +render a published file differently, and the generator refuses that too. After a report or contract change, run `python scripts/regenerate_goldens.py`, inspect the semantic diff, then run `python scripts/regenerate_goldens.py --check` and the affected behavioral diff --git a/STABILITY.md b/STABILITY.md index e05cb93d..569abe5c 100644 --- a/STABILITY.md +++ b/STABILITY.md @@ -342,6 +342,13 @@ that cannot be expressed additively needs `2.0`. A deprecation cycle is counted in **shipped releases**, never in elapsed time on unreleased `main` — a deprecation nobody could install gave nobody the chance to migrate. Every published `docs/report-schema.v.json` keeps its bytes forever. +So does every other `docs/*-schema*.json` a release tag carries: each is +pinned by sha256 in `scripts/published_schema_pins.json`, the generator refuses +to rewrite one, and a change needs a successor schema version (#1044). A +release bump no longer rewrites agent-handoff v9, agent-result v3 or +agent-boundary-result v3: their schema default for `tool.version` stays +`1.2.0`, the value v1.2.0 published (v1.0.0 and v1.1.0 each stamped its own +version there), while the field itself still reports the running version. The freeze is conditional: it holds while no breaking change lands, and a breaking change restarts it and invalidates every qualification receipt diff --git a/docs/release-runbook.md b/docs/release-runbook.md index 28fd1214..1963213d 100644 --- a/docs/release-runbook.md +++ b/docs/release-runbook.md @@ -554,6 +554,16 @@ The shape that holds: rehearsal is green: `release-rehearsal.yml` for a qualified release; for an advisory one, Release Engine Smoke and then `release-advisory-rehearsal.yml` on the same commit. + The change that sets the version regenerates with + `python scripts/generate_schemas.py` and then runs + `python scripts/generate_schemas.py --pin`, which pins in + `scripts/published_schema_pins.json` every `docs/*-schema*.json` the tag + will publish. While the version has no tag, + `tests/test_published_schema_freeze.py` fails until every one is pinned, so + nothing reaches the tag unpinned. A version bump changes no schema document + (#1044). If the generator refuses because one would change, either a model + changed a published file, which needs a successor schema version, or the + installed pydantic is not the locked one. 4. Push the tag: `git tag v0.16.0 && git push origin v0.16.0`. 5. The `verify` job runs unattended. 6. Approve the `pypi` environment gate on the `publish` job, using the readiness diff --git a/docs/report-1-0-contract.md b/docs/report-1-0-contract.md index 548866ea..b40fa10f 100644 --- a/docs/report-1-0-contract.md +++ b/docs/report-1-0-contract.md @@ -52,10 +52,18 @@ receipt collected against the frozen schema — see consumer gates on, or changing what a value means is a major. 3. **Every published schema URL keeps its bytes forever.** `docs/report-schema.v.json` is immutable once committed; new content goes to a new version. This is - enforced by byte hash in `tests/test_effect_coverage.py` - (`_PUBLISHED_SCHEMA_SHA256`), because "committed equals generated" — which - is all `generate_schemas.py --check` proves — does not prove that a content - change moved the version. + enforced by byte hash, because "committed equals generated" does not prove + that a content change moved the version. Every `docs/*-schema*.json` a + release tag carries is pinned in `scripts/published_schema_pins.json`: the + set is derived from the tags, not from a hand list, and a published file + that is missing, unpinned or changed fails `generate_schemas.py --check` and + `tests/test_published_schema_freeze.py`. The generator refuses to rewrite + one and names the successor-version route instead (#1044). No schema + embeds the running package version any longer, so a release bump leaves + them all byte-identical: agent-handoff v9, agent-result v3 and + agent-boundary-result v3, which did, keep the `tool.version` default of + `1.2.0` that v1.2.0 published (v1.0.0 and v1.1.0 each stamped its own + version there). 4. **A deprecation cycle is counted in shipped releases, never in elapsed time on unreleased `main`.** A surface deprecated in `1.n` may be removed no earlier than `2.0`, and only if `1.n` was actually published. Time a diff --git a/scripts/generate_schemas.py b/scripts/generate_schemas.py index f0c80e37..396baa4e 100644 --- a/scripts/generate_schemas.py +++ b/scripts/generate_schemas.py @@ -91,6 +91,16 @@ committed file. Drift exits non-zero with a unified diff preview, so a Pydantic model edit that forgets to regenerate fails CI fast with an actionable message. +A ``docs/*-schema*.json`` file that a release tag carries is published and +never rewritten (#1044; see "Published-schema freeze" below). When +regenerating would change one, the generator writes nothing, exits 1, and says +the change needs a successor schema version. ``--check`` also reports a +published file that is missing, unpinned, or no longer matches its pin in +``scripts/published_schema_pins.json``. ``--pin`` adds the pins a release is +missing and never changes an existing one: + + python scripts/generate_schemas.py --pin # release preparation + Tests should import ``build_*_schema`` directly — they return ``(Path, str)`` tuples without touching disk, so unit tests stay subprocess-free. """ @@ -99,11 +109,15 @@ import argparse import difflib +import fnmatch +import hashlib import json +import re +import subprocess import sys from collections.abc import Callable from pathlib import Path -from typing import Any, get_args +from typing import Any, NamedTuple, get_args REPO_ROOT = Path(__file__).resolve().parent.parent DOCS = REPO_ROOT / "docs" @@ -175,6 +189,320 @@ def _emit(target: Path, content: str, *, check_only: bool, drift: list[str]) -> return True +# --- Published-schema freeze (#1044) ---------------------------------------- +# +# A docs/*-schema*.json file is published once a release tag carries it. Its +# `$id` is a raw URL consumers fetch and pin by digest, so its bytes never +# change again: new content goes to a successor schema version +# (docs/report-1-0-contract.md, rule 3). "Committed equals generated", which is +# all the drift check proves, cannot show that, so three things hold it: +# +# - PINS_PATH records the sha256 of every published file. A deliberate +# re-freeze edits a pin, where a reviewer sees it, and is recorded under +# `refrozen` when the bytes then differ from a release that shipped them. +# - `freeze_violations()` derives the published set from the release tags +# rather than from a hand list, and reports a published file that is +# missing, has no pin, no longer matches its pin, or differs from what the +# newest release carrying it published. `--check` runs it, and so does +# tests/test_published_schema_freeze.py. +# - `main()` refuses to rewrite a published or pinned file: when regenerating +# would change one, it writes nothing at all and exits 1. +# +# Release tags come from git. Where none can be read (a shallow clone, a +# source archive) the generator falls back to the pin manifest for the +# published set rather than failing closed. That loses nothing on a commit CI +# passed: CI fetches tags and fails when a tag-published file has no pin, so +# there the manifest names every published file. The two checks that need the +# tags themselves (a missing pin, the newest release's bytes) are skipped with +# a note; the freeze tests, which need them, fail instead. +# +# While the package version has no tag yet (release preparation: the version +# is bumped, the tag not cut), every schema document on disk is about to be +# published, so every one must be pinned before the tag exists. `--pin` adds +# those pins. + +_PINS_SHOWN = "scripts/published_schema_pins.json" +PINS_PATH = REPO_ROOT / _PINS_SHOWN +SCHEMA_DOCUMENT_GLOB = "*-schema*.json" +_RELEASE_TAG = re.compile(r"v\d+\.\d+\.\d+\S*") + + +def is_schema_document(name: str) -> bool: + return fnmatch.fnmatchcase(name, SCHEMA_DOCUMENT_GLOB) + + +class SchemaPins(NamedTuple): + #: basename -> sha256 of the bytes a published file keeps. + sha256: dict[str, str] + #: basename -> the release tag whose bytes the pinned bytes deliberately + #: differ from (an owner decision recorded in the manifest). + refrozen: dict[str, str] + + +def load_schema_pins(path: Path | None = None) -> SchemaPins: + source = path or PINS_PATH + payload = json.loads(source.read_text(encoding="utf-8")) + sha256 = payload["sha256"] + refrozen = {name: entry["differs_from_release"] for name, entry in payload["refrozen"].items()} + malformed = sorted( + name + for name, digest in sha256.items() + if not (is_schema_document(name) and re.fullmatch(r"[0-9a-f]{64}", str(digest))) + ) + if malformed or not set(refrozen) <= set(sha256): + raise ValueError( + f"{source}: malformed pins {malformed}, or a re-frozen file without a pin " + f"{sorted(set(refrozen) - set(sha256))}" + ) + return SchemaPins(sha256=dict(sha256), refrozen=refrozen) + + +def _add_schema_pins(new: dict[str, str], path: Path | None = None) -> None: + target = path or PINS_PATH + payload = json.loads(target.read_text(encoding="utf-8")) + overlap = sorted(set(new) & set(payload["sha256"])) + if overlap: + raise ValueError(f"--pin never changes an existing pin: {overlap}") + payload["sha256"] = dict(sorted({**payload["sha256"], **new}.items())) + target.write_text(_canonical_json(payload), encoding="utf-8") + + +class ReleaseTags(NamedTuple): + #: Every release tag read, oldest first. + names: tuple[str, ...] + #: basename -> {tag: git blob id}, oldest tag first, for each + #: docs/*-schema*.json any release tag carries. + schema_blobs: dict[str, dict[str, str]] + + def newest(self, name: str) -> tuple[str, str]: + tag, blob = list(self.schema_blobs[name].items())[-1] + return tag, blob + + def pending(self, version: str) -> str | None: + """The tag ``version`` will be released under, while it does not exist.""" + + from packaging.version import Version + + if any(Version(tag[1:]) == Version(version) for tag in self.names): + return None + return f"v{version}" + + +def read_release_tags(repo_root: Path | None = None) -> ReleaseTags | None: + """The release tags of this checkout and the schema documents each carries. + + None when no release tag can be read: no git, a directory that is not this + repository's own checkout, or a clone that fetched no tags. + """ + + from packaging.version import InvalidVersion, Version + + root = (repo_root or REPO_ROOT).resolve() + + def git(*args: str) -> str | None: + try: + completed = subprocess.run( + ["git", "-C", str(root), *args], capture_output=True, text=True, check=False + ) + except OSError: + return None + return completed.stdout if completed.returncode == 0 else None + + toplevel = git("rev-parse", "--show-toplevel") + if toplevel is None or Path(toplevel.strip()).resolve() != root: + return None + listing = git("tag", "--list", "v*") + if listing is None: + return None + versions: dict[str, Version] = {} + for tag in listing.split(): + if not _RELEASE_TAG.fullmatch(tag): + continue + try: + versions[tag] = Version(tag[1:]) + except InvalidVersion: + continue + if not versions: + return None + names = tuple(sorted(versions, key=versions.__getitem__)) + schema_blobs: dict[str, dict[str, str]] = {} + for tag in names: + tree = git("ls-tree", tag, "--", "docs/") + if tree is None: + return None + for line in tree.splitlines(): + meta, _, path = line.partition("\t") + _mode, kind, blob = meta.split() + name = path.removeprefix("docs/") + if kind == "blob" and "/" not in name and is_schema_document(name): + schema_blobs.setdefault(name, {})[tag] = blob + return ReleaseTags(names=names, schema_blobs=schema_blobs) + + +def _git_blob_id(data: bytes, *, like: str) -> str: + """Git's object id for ``data`` as a blob, in the hash ``like`` uses.""" + + header = b"blob %d\0" % len(data) + digest = hashlib.sha256 if len(like) == 64 else hashlib.sha1 + return digest(header + data).hexdigest() + + +class FreezeViolation(NamedTuple): + #: missing | unpinned | changed | differs_from_release + kind: str + name: str + message: str + + +_SUCCESSOR_ROUTE = ( + "A published schema keeps its bytes forever (docs/report-1-0-contract.md, " + "rule 3): put the change in a successor schema version and leave this file " + "as it is." +) + + +def freeze_violations( + *, + docs: Path | None = None, + pins: SchemaPins | None = None, + tags: ReleaseTags | None, + version: str, +) -> list[FreezeViolation]: + """Every way the published schema documents in ``docs`` break the freeze. + + The published set is every file a release tag carries, every pinned file + and, while ``version`` has no tag yet, every schema document on disk. With + ``tags=None`` only the pins can be checked. + """ + + docs = docs or DOCS + pins = pins or load_schema_pins() + published: dict[str, str] = {name: "pinned" for name in pins.sha256} + pending = tags.pending(version) if tags is not None else None + if tags is not None: + for name in tags.schema_blobs: + published[name] = f"published at {tags.newest(name)[0]}" + if pending is not None: + for path in docs.glob(SCHEMA_DOCUMENT_GLOB): + published.setdefault( + path.name, + f"about to be published at {pending}: the package version " + "has no release tag yet", + ) + violations: list[FreezeViolation] = [] + for name, where in sorted(published.items()): + path = docs / name + if not path.is_file(): + violations.append( + FreezeViolation( + "missing", + name, + f"docs/{name}: {where}, but missing. A published schema URL " + "never goes away.", + ) + ) + continue + data = path.read_bytes() + pin = pins.sha256.get(name) + if pin is None: + violations.append( + FreezeViolation( + "unpinned", + name, + f"docs/{name}: {where}, but has no pin in " + f"{_PINS_SHOWN}. Add it with " + "`python scripts/generate_schemas.py --pin`.", + ) + ) + elif hashlib.sha256(data).hexdigest() != pin: + violations.append( + FreezeViolation( + "changed", + name, + f"docs/{name}: {where}, and its bytes no longer match its pin. " + + _SUCCESSOR_ROUTE, + ) + ) + if tags is not None and name in tags.schema_blobs: + tag, blob = tags.newest(name) + if _git_blob_id(data, like=blob) != blob and pins.refrozen.get(name) != tag: + violations.append( + FreezeViolation( + "differs_from_release", + name, + f"docs/{name}: differs from the bytes {tag} published " + f"(`git diff {tag} -- docs/{name}`). " + _SUCCESSOR_ROUTE + + " Keeping these bytes instead is an owner decision, " + "recorded under `refrozen` in the pin manifest.", + ) + ) + return violations + + +_RELEASE_TAGS_UNAVAILABLE = ( + "note: no release tag can be read here (a shallow clone or a source " + "archive), so the published set is the pin manifest alone, " + f"{_PINS_SHOWN}. CI fetches tags and checks the " + "manifest names every file a release carries; `git fetch --tags` to check " + "it here.\n" +) + +_PYDANTIC_HINT = ( + "If no model changed, compare the installed pydantic with constraints/dev.txt: " + "the committed schemas are rendered by the locked pydantic, and another " + "version renders some of them differently (#1088)." +) + + +def _frozen_rewrites( + outputs: list[tuple[Path, str]], *, pins: SchemaPins, tags: ReleaseTags | None +) -> dict[str, str]: + """Each published or pinned schema document regenerating would rewrite, and why. + + The bytes a frozen file must keep are its pin, or its committed bytes while + it has none yet. + """ + + frozen = set(pins.sha256) | (set(tags.schema_blobs) if tags is not None else set()) + refusals: dict[str, str] = {} + for target, content in outputs: + name = target.name + if name not in frozen or not is_schema_document(name): + continue + generated = content.encode("utf-8") + pin = pins.sha256.get(name) + if pin is not None: + if hashlib.sha256(generated).hexdigest() == pin: + continue + kept = target.read_text(encoding="utf-8") if target.is_file() else "" + elif target.is_file(): + kept = target.read_text(encoding="utf-8") + if kept == content: + continue + else: + kept = "" + where = ( + f"published at {tags.newest(name)[0]}" + if tags is not None and name in tags.schema_blobs + else f"pinned in {_PINS_SHOWN}" + ) + diff_lines = list( + difflib.unified_diff( + kept.splitlines(keepends=True), + content.splitlines(keepends=True), + fromfile=f"docs/{name} (kept)", + tofile=f"docs/{name} (generated)", + n=2, + ) + ) + refusals[name] = ( + f"docs/{name}: {where}; regenerating it would change its bytes. " + f"{_SUCCESSOR_ROUTE} {_PYDANTIC_HINT}\n" + + "".join(diff_lines[:_DIFF_PREVIEW_LINES]) + ) + return refusals + + def build_manifest_schema() -> tuple[Path, str]: from agents_shipgate.schemas.manifest import AgentsShipgateManifest @@ -3036,24 +3364,86 @@ def main(argv: list[str] | None = None) -> int: prog="generate_schemas", description=( "Regenerate docs/*.json schemas (default) or verify they match the " - "current Pydantic models (--check)." + "current Pydantic models (--check). A published schema document is " + "never rewritten: a change to one needs a successor schema version." ), ) - parser.add_argument( + mode = parser.add_mutually_exclusive_group() + mode.add_argument( "--check", action="store_true", - help="Verify committed schemas match the generators; exit 1 on drift.", + help=( + "Verify committed schemas match the generators and published ones " + "match their pins; exit 1 on drift." + ), + ) + mode.add_argument( + "--pin", + action="store_true", + help=( + "Pin every published schema document that has no pin yet; while the " + "package version has no release tag, that is every schema document. " + "Never changes an existing pin. Needs release tags, and refuses while " + "anything drifts." + ), ) args = parser.parse_args(argv) - DOCS.mkdir(parents=True, exist_ok=True) - (DOCS / "examples").mkdir(parents=True, exist_ok=True) + from agents_shipgate import __version__ + + outputs = [builder() for _name, builder in BUILDERS] + pins = load_schema_pins() + tags = read_release_tags() + if tags is None: + sys.stderr.write(_RELEASE_TAGS_UNAVAILABLE) + refusals = _frozen_rewrites(outputs, pins=pins, tags=tags) + + if not (args.check or args.pin): + if refusals: + sys.stderr.write("\n".join(refusals.values())) + sys.stderr.write( + f"\n\nRefused: regenerating would change {len(refusals)} published " + "schema file(s). Nothing was written.\n" + ) + return 1 + DOCS.mkdir(parents=True, exist_ok=True) + (DOCS / "examples").mkdir(parents=True, exist_ok=True) + for target, content in outputs: + _emit(target, content, check_only=False, drift=[]) + return 0 + drift: list[str] = [] - for _name, builder in BUILDERS: - target, content = builder() - _emit(target, content, check_only=args.check, drift=drift) + for target, content in outputs: + if target.name not in refusals: + _emit(target, content, check_only=True, drift=drift) + violations = freeze_violations(pins=pins, tags=tags, version=__version__) + frozen = list(refusals.values()) + [ + violation.message + for violation in violations + if not (args.pin and violation.kind == "unpinned") + ] - if args.check and drift: + if args.pin: + if tags is None: + frozen.append("--pin needs the release tags: run `git fetch --tags`.") + if drift or frozen: + sys.stderr.write("\n".join(drift + frozen)) + sys.stderr.write("\n\nNothing was pinned.\n") + return 1 + new = { + violation.name: hashlib.sha256((DOCS / violation.name).read_bytes()).hexdigest() + for violation in violations + if violation.kind == "unpinned" + } + if not new: + print("Every published schema document is already pinned.") + return 0 + _add_schema_pins(new) + for name in sorted(new): + print(f"Pinned docs/{name}") + return 0 + + if drift: sys.stderr.write("\n".join(drift)) sys.stderr.write( "\n\nSchema drift detected in " @@ -3061,8 +3451,13 @@ def main(argv: list[str] | None = None) -> int: " python scripts/generate_schemas.py\n" " git add docs/ && git commit\n" ) - return 1 - return 0 + if frozen: + sys.stderr.write("\n" + "\n".join(frozen)) + sys.stderr.write( + f"\n\n{len(frozen)} published schema problem(s). Regenerating does not " + "resolve these: a published file keeps its bytes.\n" + ) + return 1 if drift or frozen else 0 if __name__ == "__main__": diff --git a/scripts/published_schema_pins.json b/scripts/published_schema_pins.json new file mode 100644 index 00000000..099b1906 --- /dev/null +++ b/scripts/published_schema_pins.json @@ -0,0 +1,192 @@ +{ + "about": "sha256 of every published docs/*-schema*.json: the bytes it keeps forever (#1044). scripts/generate_schemas.py derives the published set from the release tags and enforces these pins. Add pins with `python scripts/generate_schemas.py --pin`; change one only as a deliberate re-freeze, recorded under `refrozen` when the bytes then differ from a release.", + "refrozen": { + "agent-boundary-result-schema.v3.json": { + "differs_from_release": "v1.2.0", + "why": "#1088 (the pydantic 2.14 lock refresh) closed CapabilityDiffRow with `additionalProperties: false` after 1.2.0, recorded in CHANGELOG; #1044 froze these bytes rather than restoring 1.2.0's." + }, + "verifier-schema.v0.21.json": { + "differs_from_release": "v1.2.0", + "why": "#1088 (the pydantic 2.14 lock refresh) closed CapabilityDiffRow with `additionalProperties: false` after 1.2.0, recorded in CHANGELOG; #1044 froze these bytes rather than restoring 1.2.0's." + } + }, + "sha256": { + "agent-boundary-result-schema.v1.json": "14e04207069c91291400a28359d4837663ebf413b8b610ca541824f93ba28d6e", + "agent-boundary-result-schema.v2.json": "179f849080fabdc59cdf4b86b5ec6a0d9c605e1ac31eb130ee2463c6a0ab361d", + "agent-boundary-result-schema.v3.json": "d146352078d76f91b79f869c76cb57648fdf5b033e36d4a3dcc667ce86b5893b", + "agent-control-schema.v1.json": "4f28e47bc37deffe2bf9103d9c28bc92809f332dd914c778df6308cfc2dd5e71", + "agent-handoff-schema.v1.json": "041fcd2958f4a6b8fd3e4b558cfcd6152dc9dabe4ab78ba304fb9138f0e12585", + "agent-handoff-schema.v2.json": "08c78edc8dbc7f2b0d773f0d3996189d332bdb21962666580161cc9a38a0fac9", + "agent-handoff-schema.v3.json": "5ed828e48ff49fffe03c1001975f12da52e55efb4baf8a585a09466c60ab80cf", + "agent-handoff-schema.v4.json": "54aae51f5950a2844ab964592c9c50e9a6f4fd5faf1bb3951c811c08b7fc1e0f", + "agent-handoff-schema.v5.json": "e090948b94bad460d7ae0aa248eb93fba5b5a3b4dcc5fc0a0d7ecc8ff302fc42", + "agent-handoff-schema.v6.json": "9dafcfe06ef2abce3a8ecd5cd508000372091160a3cd9cc6eb2ca0571ff19d62", + "agent-handoff-schema.v7.json": "d40bc7841c4060d44f84c10bb4221f42872e076d7ea47e8bfde5abd8f9dbf8a6", + "agent-handoff-schema.v8.json": "036dc757914a297b34ebb9b7ca10c21869c5c91820fa7f07bda415c1effe30c3", + "agent-handoff-schema.v9.json": "e3a6e065e4367da6898515184f429f2da404fd6d76824c41e683a2b9bb4612b0", + "agent-result-schema.v1.json": "9e810a4b3dd5728565e1b4972b916c7c4ace865a5fa65a8887b5cf9bf59f20e5", + "agent-result-schema.v2.json": "450930266912a33a74a9b52fd84bee7c300b190b0b3941979607128aef8b77c3", + "agent-result-schema.v3.json": "de2fecd92a603e5dcc8b0ad5143d02f5ea05bc0d667fe295c18c68e36dc007de", + "agent-trace-event-schema.v0.1.json": "66d41daa48778b87159c9017d9e62640fb4dbf86524448bb1eeb1191b30ee0b6", + "agent-workflow-evidence-bundle-schema.v0.1.json": "d8900202f156a81a4d87a8f8a506b939d695894cba40a262597b54fc9e919db9", + "attestation-schema.v0.1.json": "73177b57ba9a7a8de1bc1594b33a358b19f589418fabd6c9371c1ed5c93f7a69", + "attestation-schema.v0.2.json": "97da3c2ab4ed450a42ee255eaee19806ac61e6fc7c8654c6ee2418d0bb36a2d8", + "attestation-schema.v0.3.json": "46d3429f5a35cd5ca217c1db2c85d51fed09dabbe135db9819140f16a89422ed", + "attestation-schema.v0.4.json": "0b0354b487e7d450321f2db1eb50a429b27d9a542ce55d2a5a81f64bc31e911d", + "attestation-schema.v0.5.json": "499b760dc02c72909bd1b039961088708b93d636db05222e8c1ec615d1f8ce82", + "capability-delta-attestation-schema.v1.json": "3a32b2f6af56aa6ac967fedbb7d872cbbf5ebb54738f6f1794aabce0c9e867c7", + "capability-lock-diff-schema.v0.3.json": "e70a056a5c8bbe95a499d70abe3cc8017110c731e3c0db7be7ef6e9d1b319736", + "capability-lock-diff-schema.v0.4.json": "86fa7867fb94d7e8e516db382d2ecb103fef021ce7792d92cd327a80e8e17d8d", + "capability-lock-diff-schema.v0.5.json": "6492bfef0ecda7d1782140e02b13fa8bdef417c0b96561b224e474142f9afedd", + "capability-lock-diff-schema.v0.6.json": "63b63320c6c2be72b0d040e51cdb845c156e59eb07e797c0ab39243450ba290c", + "capability-lock-diff-schema.v0.7.json": "e73210870bb5b1181fcbb90872ced3c71f3ecc1f11124188aa21579694eec93a", + "capability-lock-diff-schema.v0.8.json": "c872af71d41a445d1e28065704e034b2a0a32c88a5ec9e767ad8853f4373ac20", + "capability-lock-diff-schema.v0.9.json": "9dd637b6bcbb9d3f53187e7392ff3ff17faf31ae3ccd97852e9f555a24777ed2", + "capability-lock-schema.v0.1.json": "105439eff2442885d0ac60aa0fd921e9c9cba52b2596ed2a410835d177802e9c", + "capability-lock-schema.v0.2.json": "f87d1b666cb500c96e9267e6e75146a2b0d69b9c051b751d7c37c6419eb02932", + "capability-lock-schema.v0.3.json": "3e3c70cbdcf9af087f0fa4b7d73eaccbd8574ee456c8e03d225cc12c1393d19a", + "capability-lock-schema.v0.4.json": "1c5a603659a076eb273476332fabe93736eeb43392d233a299079c1f703eaaea", + "capability-lock-schema.v0.5.json": "5282c9c5e78b8f2ae5f1ceb38e2a580c19c69bc1f670b2af596c18a2f2ecce41", + "capability-lock-schema.v0.6.json": "0e43ccadea3258323cac279a59a431e771c2abbe9f4333c853ec5d1401f5285a", + "capability-lock-schema.v0.7.json": "a90789c318f4a4c19347be9fc440d32322f9befa0c94461d10fb718f5a276704", + "capability-lock-schema.v0.8.json": "0900e6225c667619aaf1d5609cd45dd0bc2a44ed4c3b20ae56b42edece0b3105", + "capability-payload-schema.v1.json": "26de1cce200ad9b5552e13e34698f35be7d6e64c0cf77fa4a90d9630c1699e68", + "codex-boundary-result-schema.v1.json": "818fde799c60f0b1391241f0bb0f2b34a8faa122f2505e9f64cb591d40fa3723", + "codex-boundary-result-schema.v2.json": "7134a8b1e2c03f0ff8669cac36edb8c3a8e006c4b58d214d9b37cd7ddc4fa0be", + "current-control-schema.v1.json": "b02265de682f154a5b5c0f93f21afd5d05aad038ba85d97acda4c5224a109f07", + "feedback-schema.v0.1.json": "9239c1bfda09f100b9fb35130a0e7cdb6b1c14924cb760c7ded7093bbcc78079", + "governance-benchmark-catalog-schema.v0.2.json": "48765571e1a1d3efdff19664c261704f443e69f1659aeee3365bf2f827fc6fdb", + "governance-benchmark-result-schema.v0.1.json": "bd8a85f414c45f2c359a95ff3fc868acf6823fe9309c31cf6ca351fb8322bf06", + "governance-benchmark-result-schema.v0.2.json": "aba10035073436f7a17a58fe4cfaf067d8a82379d33372f1648f765f29b8f2d1", + "host-grants-baseline-schema.v0.2.json": "1045f8ab9b72efcb6f34075280caa3061fed3b68f204b312795f7babb24ea02a", + "host-grants-baseline-schema.v0.3.json": "f59f2d8e8202dc86117c749758079a5334fbe874d765cde0c09aa8c6b4a51613", + "host-grants-baseline-schema.v0.4.json": "c5208779c378ce48a540eedfd6cd9f26a128aee209e3564f6b888315a71284b8", + "host-grants-baseline-schema.v0.5.json": "dc1a21097cd4d96ade48bde5d9446cff7435d82e6603b2793e7d0dad2f7a1e3f", + "host-grants-baseline-schema.v0.6.json": "61d54f6c48d1394095c52955aea6e87071b8eeec38623b988f08e916e7734471", + "host-grants-baseline-schema.v0.7.json": "b33b3c53f973d672362ceb4a7a92261c504e516ae3fb9b0b96b0672c7f79fbea", + "host-grants-drift-schema.v0.2.json": "7b189368dd56af1ab12353da7739c2b82a76df13ab2679e5bdeb5fa116322d03", + "host-grants-drift-schema.v0.3.json": "79ff00678ca0a3c0a32bb5cc11a101a3fb351d408592833c4c66dd3c08e74e60", + "host-grants-drift-schema.v0.4.json": "e75713d85bd71281a5c28d24a62ebfd0ebea06b7407af2ca97dd3f50da785dd3", + "host-grants-drift-schema.v0.5.json": "53fd5ea8d13875b431aa8446f17415926f7aaaa4e73ee2cf97202243012794c9", + "host-grants-drift-schema.v0.6.json": "aefe4a383480cf279ca2b37c774fbe9c5d50c70b563ef6573a10774de8e98eb6", + "host-grants-drift-schema.v0.7.json": "61e09a530660fdd1e936ba4da68b4ae686da45b64818052459cedcdae52f8702", + "host-grants-inventory-schema.v0.1.json": "5bf69f78d6ed6a566d39a0f0e70c19df99af69c3919dc289cda69a54b77dc9ce", + "host-grants-inventory-schema.v0.2.json": "78b736f9c91311dbadedeba632819bb816a5d1f5acbfe9591e05ef763abfdea1", + "host-grants-inventory-schema.v0.3.json": "0e0cd185f620fb462d88a577054bd996d71d58592ce72ff1526fc6a6df8d823f", + "host-grants-inventory-schema.v0.4.json": "5afcda387659e62d9c10389c4f48c816d124d301f526722ac91aeabf4798acbe", + "host-grants-inventory-schema.v0.5.json": "3c079130bc8d4d8855f3e4baddf52a277533796e959135325495ce2562b85a74", + "host-grants-inventory-schema.v0.6.json": "5471843d40e42039184f7313151d6dc1661ae1f6fca9082c89425a238b0d8012", + "host-grants-inventory-schema.v0.7.json": "5c0addc05b05cec5af55854108f6906ab918412d9876b4d397e1cb646b21a3a6", + "human-authorization-schema.v1.json": "306387ae600d84413629e8f4c4501ef33d58e6b2f9eef2602bc7258864350c36", + "human-review-decision-schema.v1.json": "c2305a255f589d8bda1604a9e3d68429c814458b948b4285349639fce203f620", + "human-review-request-schema.v1.json": "0f39c003b4a99facd4808c890c9acb4186e18455a774ad7c1a8d05417504b828", + "org-evidence-bundle-schema.v1.json": "5fd0a5f8bdc44ebe61fd4c87c8455662d3fd51daef5d79242f589d2f361741cc", + "org-evidence-bundle-schema.v2.json": "5522c0faf05e40d42b906a4e0bf29aed7682b23c8d229e29acb5609a91f79572", + "org-governance-schema.v0.1.json": "0d5e601ab13e35c96bed5c0770433804eca858eb787fddc70db46a222f7795fa", + "packet-schema.v0.1.json": "da332bd634e04cc56996d8cd325afbb8590cf4ddfa58d06d2aeeda1a6f084594", + "packet-schema.v0.10.json": "dd79f3518f933957e03fd6588dac16915c1add2c53cc45eda3673c0895838c69", + "packet-schema.v0.11.json": "248727f8baeb667361d01b0936d32fc418d0f8cad197dc64e268dc21d104593c", + "packet-schema.v0.12.json": "8bef394df037374a14153b743f46a4e279f65b8e1312842efcdec92a07d0bcb5", + "packet-schema.v0.13.json": "80e6665a4a4d4778b2259c96f756983b8f6857ebca3cb06ecf724c80cfae06eb", + "packet-schema.v0.14.json": "7bb394413688adcdbbc7f4c599fd361d0c1280c3bf7a049a72d782a7852b85f8", + "packet-schema.v0.15.json": "b6a47a71482592732c2cac0120a154ce39a6830ed06cc457f6c1fdbb842bdead", + "packet-schema.v0.16.json": "47c3aee2737985c63899bd110f6805fdcd40172e47251fd215342bc728351e06", + "packet-schema.v0.17.json": "269f7752f6cbe5b9097b259d18939ea64a06250e0802cf6ee6b2f655432164e8", + "packet-schema.v0.18.json": "90130c9aba38a50e8bddaac8646f380888e20dd50fcc8b18632137d291fcf331", + "packet-schema.v0.2.json": "a167b2a03d538646e1565bbb9ca69174dd54f9703d681ecf11e953fc3a47044d", + "packet-schema.v0.3.json": "effd0a67d48d1cf358b9d9474ef359085a76fbff31a152b90a102c93f90bb2ab", + "packet-schema.v0.4.json": "956e53804ce66d57fbb4ac417e7b759e95fbe15f037db7bff394be90b905e30d", + "packet-schema.v0.5.json": "6bf1955ca24355b986cebb5e8b8f5d9be289c672975f36e097e53b28a5a8727f", + "packet-schema.v0.6.json": "5885a71629a784f42a122c165e359a11d03d72dd638ebb75ad775c87df3f85c9", + "packet-schema.v0.7.json": "aebfc30cb0da6f9064ce8744f82afa41fbd2c6e65f9209325a45108de9ad2cdf", + "packet-schema.v0.8.json": "38805f71e96eb17a7fb7f4a8ce26fc99d02e06c0d114c14a6eef098aa036f860", + "packet-schema.v0.9.json": "7ab269dd3ae2d379f2f771845ef0463775017bce42ec0b406aeab993fdea553e", + "policy-pack-schema.v0.1.json": "fbbe4a9feb5be38963f402c2423979c9e58ba15c0a1449bd34ce2342275d3723", + "policy-pack-schema.v0.2.json": "a99781ccb42b69e0e35bce5ecf74ecfac1e9dbaef3d01d2e03af1238e1f61b2c", + "policy-pack-schema.v0.3.json": "70892ce8aee7c72ab762a85da622ec383f9663728830c2c775e9e8ebd90d7c25", + "policy-pack-schema.v0.4.json": "a706e0d1f55c8781482e04ef07b265259497a14ed58786ffaa4a2e7ee39557d0", + "preflight-schema.v0.1.json": "32d997b737344e6526315160253cb4a9836681b15ec000a971c02f53d9481fc2", + "preflight-schema.v0.2.json": "8b53341d0d5e1e04070907ed5f93c740421f6ef04aae82efeca34bd5a2446da9", + "preflight-schema.v0.3.json": "a14904c40596a3fd1685fbf0e69ecd8a9ee3276cbfa316e0c4851dcb74aa8b7c", + "preflight-schema.v0.4.json": "048c4785253afa476e7b86f6175d61328ea299641f6253c3d4315c9d61d67583", + "preflight-schema.v0.5.json": "b85dfd1d1ba84fadb87c739c147ec49d2d329f405f0f0cd2140c1c35667a7947", + "registry-schema.v0.2.json": "8330abd6b46b157720770d7a7bb5f2e26e5e7299af6bc15df4236362bc2eafa8", + "registry-schema.v0.3.json": "b1010787a24cf986a51487a8b93dfd66ecdfb72e80a14256112ee28c1251462d", + "registry-schema.v0.4.json": "16586274a39150198eab1f7a00c1ff90fef3c3927ab39e5f7e033c6a69327963", + "report-schema.v0.1.json": "08f31d29075b3b3ded22f6c293cfe081437ccc6aa6eafd7873dbdef4e29e5770", + "report-schema.v0.10.json": "4e708d7e93e7991c5781901964fbc36959ff61779748ce016e821446e88ace9d", + "report-schema.v0.11.json": "bf546f2dcbc154a8ac88d38448677feef1a2f15ea0ed2d2efc5d3e809b096778", + "report-schema.v0.12.json": "6bcfde31130c4bd9971c700411ec17c05f5d14e6d13d6b7b20879f980f16ad3d", + "report-schema.v0.13.json": "f4f1e3a94f5eed481dd1780c6d842564b4d7aec577b6420b9190d51abd5928da", + "report-schema.v0.14.json": "5e5b229d6a12ac6d75e907c232ca67e974e9ffbd55f103e9ce47d4d9a5618013", + "report-schema.v0.15.json": "d4ca2c63fd14a4de7d00e09dee563b258a77fd954f20f262d3c40a32c41f7ee7", + "report-schema.v0.16.json": "d6a7f78a9f10589d4b04870ce80fd0453c7dcf8950768c3833a6b88b4be3a67b", + "report-schema.v0.17.json": "f32c0ad2a332dc8e836f236c67d3492664b5588e994bdd15abeebb916c71ee7a", + "report-schema.v0.18.json": "48c015f7960f68009d186c77c9b3043618f7e1faeb2515747fe97ecd45099cf0", + "report-schema.v0.19.json": "2167155966ace52359131549be16b3af84cc3e0ec4f0128cda4e3c6c63f59288", + "report-schema.v0.2.json": "8d0352386d21c628fabe8d5e268aaa41c8a088ba3cc6721447e9e0f335525702", + "report-schema.v0.20.json": "a3fd07215aed372078157d1fa89688fd00803cb53a4a2f4b7ad695811ca10b8a", + "report-schema.v0.21.json": "7919e4fdcf466ae4c33c4fc245c07626b81614dafc54cd28cdc2019f351c1694", + "report-schema.v0.22.json": "0159b27cc8f8a858ca27727ecd72abc20fa04ba24799471d79982de5dbca4571", + "report-schema.v0.23.json": "081f565b0b159e1115ca220bbc177e6c1adbe3d04777d3a1ea2de0f3c3f07e98", + "report-schema.v0.24.json": "470724e4fa1578ed1a66cd242baa648df4d5962ca23befc071607265c9ee50d3", + "report-schema.v0.25.json": "b79547b5812434ef991073d48702fc0af392339805060a622a3d69c75d9e8913", + "report-schema.v0.26.json": "740830558fc646423d3d0f037d6ed9f71863e662cc732652fa2833cb19c004b7", + "report-schema.v0.27.json": "f68131c86b9e1d5a2acc5608ef9fbe9fec7879d652b3a5467199ef4b10c9fd67", + "report-schema.v0.28.json": "f9a6f8904c08192355edc191c387b1ca01b64a5526380436a61b2b3c2ebd7d60", + "report-schema.v0.29.json": "e861b8ba0d1b9c1288e16e43a066fbc9303dd8a0d70883818102dd4421579279", + "report-schema.v0.3.json": "ef38e29063258b3e35840b0430fcbddd86afd792ca88afab1bf963dc0d0eec4e", + "report-schema.v0.30.json": "a84c09b31cf95d9cfa432e77533a5fc636dbd86f53b5e0cb90b3817ad4412d9b", + "report-schema.v0.31.json": "8bc29897d9ddc993db2ac494bb63146f0d3c314eede9ea2d9362da9595d61ad8", + "report-schema.v0.32.json": "28116e0700306789b6d502b0f892a0a23afc86f29b9e0b9e6f35eca163a91c3e", + "report-schema.v0.33.json": "e0e0e6113b86a6a070f32c5f0e5f09c04af88405ff937057a10250f63c48a185", + "report-schema.v0.34.json": "29c9d15e49d84ac4acae9b43842947dad8c7440accfd2283e50b4f0c67d415e9", + "report-schema.v0.35.json": "cd3a971dd6a02676cfa0db798a443715778b364bd80510b23f3c83ea876003cf", + "report-schema.v0.36.json": "c9c0d7576b23fbacfff2da390d462b66add673f448c28eb8295938cf2f986308", + "report-schema.v0.37.json": "c5a8551abfab7a95f3c40b2470eb687a2fc95f4c572da318d5a38cfc91933b43", + "report-schema.v0.38.json": "df00a7abfbd59fa3274caacd78acf5960f8c754fab2d6e1a62840b7997ca8d24", + "report-schema.v0.39.json": "9a150c97a13c59b4f1ba68d86d4393be1c799e6c84d1638acee825e28646dab9", + "report-schema.v0.4.json": "881de7c6fcf359212bbef9f581e8a9618c3e7192d5b4014a0b9c3a892493d8d0", + "report-schema.v0.40.json": "eee3c02e3bdb25d12c2e3ec46a2c72407f735252f1e6d0804e23c43cf5e86aff", + "report-schema.v0.41.json": "2d201cc9767ae8707c1c14dc9a56d49b65429d5d7c32ad6afc30b2c4d66f879b", + "report-schema.v0.42.json": "3f149478eb62809d3a40de060269520395ca763496a57ec0bc4a8d37e9b0076e", + "report-schema.v0.43.json": "e3d13aca31480e170e1c6656a4f075191ddc4da747cf00f5c5c7066d266ac7bd", + "report-schema.v0.5.json": "df76f880ca39b372b8c0c2a9f2db1c2af53e847641a66ca690abc78675acd276", + "report-schema.v0.6.json": "b13e60764c0b372c84598d2034c04d998ed603312ea0f6fb4a1408f7b1ce5ba2", + "report-schema.v0.7.json": "6a8e0dd182ffc76b7101d32ba3669f4841bc55cbff5552e94b66a72280253e9b", + "report-schema.v0.8.json": "d19d99a6da42c0cc77760546c133251aa558a9478d956e2b34ebcb2f7fae3be6", + "report-schema.v0.9.json": "9b91a29c8c38d9eb0d3b31b0771ad45482c528c56f8c31c6ae614987c9eb4ab5", + "report-schema.v1.0.json": "d010e784457a5335875bae3d957a60e4a5623a4484ee12361599cf2775a04784", + "scenario-schema.v0.1.json": "58e1386c2c6edeb7a3e73248fe7c0fb1179114192c23384fd270a2de20a79289", + "verification-artifact-manifest-schema.v1.json": "abe97ad493878f9c8953478851c541fa51c5a695deda05d53168281d3b2a62d4", + "verification-plan-schema.v1.json": "e70f0913202013388e7a61cbbeb395f01110151cb580b14371f3c60154e92da8", + "verification-receipt-schema.v1.json": "b5da0f75089028e751ceabebec8af07bb6f9eb055fc2f838e5ca07d8ecb5afdd", + "verification-unit-result-schema.v1.json": "af6467fd5aedc501930cbbe9af0bf9c0e6c9b2eecea8720f57ede1b62c290805", + "verifier-schema.v0.1.json": "f5c924684604dc09cfa3e467a06dc63eeb8d89a41448a96bac2ce1e6977d2989", + "verifier-schema.v0.10.json": "c582772273cadcb8abd2137b03486528d681093e088a7284e754a8c75f92e727", + "verifier-schema.v0.11.json": "adbbcd85f870b8f12f060c02b090395a34ffec4f660f1994d6001a2728397090", + "verifier-schema.v0.12.json": "fae9348ede7e04cd786f2104c55b80b4a9c623c69af1351dc6bca15af2037736", + "verifier-schema.v0.13.json": "e692bf97e2880efab8f8d480da453d6094112664e109548cc4f26e48031f00a7", + "verifier-schema.v0.14.json": "02e88f43705d8d77091686a4c77f09ce45ec532fe7fe2654674c590a3dfa0779", + "verifier-schema.v0.15.json": "f362d0bb211830d4900fca2e2f3d38e95256a9e15338ff992a88a03640f29ee5", + "verifier-schema.v0.16.json": "86c674e1d454c1541c8f217dc2bd716b0076b2bd6612f66e75bb3a3a2e8d755a", + "verifier-schema.v0.17.json": "779d21a46afe5297d1e3c29404da2172d0b4d7e6d942691d9903cfcc9b1b4efe", + "verifier-schema.v0.18.json": "42f91439d63dedd58be9350c0c1755f00c45d2beb7ffbcba301b8fef0ed4af48", + "verifier-schema.v0.19.json": "b466eba71f286518005d61e82db6a8f908134213acbee9b6a44ce4c4ffc09247", + "verifier-schema.v0.2.json": "2f9b5169f05f4c94305a0b8c362256e32c516d064f8ef5fc81c24f4ed95978f8", + "verifier-schema.v0.20.json": "ce9a71bbca4b97c7b05fd2de90ab2ca3ec5c309c947c7b3c02cbd3cf93aa8ee8", + "verifier-schema.v0.21.json": "fc03d4cdafe3189eca8894f0758c4bac3c122af253bf2423b590e352de713716", + "verifier-schema.v0.3.json": "855c346d23cdfb7ca37051e0531afaa226a5dd00397e1c059d73f51341d8e680", + "verifier-schema.v0.4.json": "fb16f089dc071271bcb3578af106f13c2cc23c8e5868c430d85d0ce91efdd558", + "verifier-schema.v0.5.json": "1616ef8e954c0cacf30ee507cae6e43257e663b2962c344855ec1c851bb380bc", + "verifier-schema.v0.6.json": "6e9044eb1b76fabe609a7042e980b361f1043f1735b03c3c569039fc86add0ad", + "verifier-schema.v0.7.json": "1ac81c16c8edb62abf49e8f7278ec89f1626185618ae90c2cc5d06913605f501", + "verifier-schema.v0.8.json": "f6aa6f2a44106f6b583a3456d3ef2a97652acbd0fd9e9625526740da60b5c85a", + "verifier-schema.v0.9.json": "d477db7c202ba9c0629fa28ade7a69d37758052de370aa3246135ede9eeefeaf", + "verify-run-schema.v1.json": "b1cb70fca8a4c494fc6968bc5fcf42b5336162d41adb69884e1e95f0ec7b70da", + "verify-run-schema.v2.json": "fbb9fad8994ba829c55b5f1a931a99cc69553d0fd8069762073332ab080e8f9e", + "verify-run-schema.v3.json": "6bad0d1751fdfabd48c04fab205d522b00ddc589fa38c26b4852cd4e13e1c33c", + "verify-run-schema.v4.json": "0423c6959004ad127dc6b4e4da21b68c75b7c66871a20dd6f7d4b3b726e5b616", + "verify-run-schema.v5.json": "19deb3ba50d3f610325b6e7457ad000ccbcbb737e03cb8bb0e8011c69343f141" + } +} diff --git a/src/agents_shipgate/schemas/agent_handoff.py b/src/agents_shipgate/schemas/agent_handoff.py index a2240f3d..48721a02 100644 --- a/src/agents_shipgate/schemas/agent_handoff.py +++ b/src/agents_shipgate/schemas/agent_handoff.py @@ -19,11 +19,29 @@ RemediationPlanSafety = Literal["allowed", "forbidden", "patch"] +#: The JSON-schema default of ``tool.version`` in the published +#: ``docs/agent-handoff-schema.v9.json``. The field's default used to be the +#: running package version, so every release rewrote the v9 file's bytes under +#: an unchanged schema version (#1044). Those bytes are now frozen as published: +#: the runtime default is still the running version, and only the schema keeps +#: this constant. A successor schema version must emit no release-specific +#: default: when v10 replaces v9 in the generator, drop ``json_schema_extra`` +#: below (v9 is then no longer regenerated, so its bytes cannot move). +_PUBLISHED_V9_TOOL_VERSION_DEFAULT = "1.2.0" + + +def _running_version() -> str: + return __version__ + + class AgentHandoffTool(BaseModel): model_config = ConfigDict(extra="forbid") name: str = "agents-shipgate" - version: str = __version__ + version: str = Field( + default_factory=_running_version, + json_schema_extra={"default": _PUBLISHED_V9_TOOL_VERSION_DEFAULT}, + ) class AgentHandoffSubject(BaseModel): diff --git a/src/agents_shipgate/schemas/agent_result_v1.py b/src/agents_shipgate/schemas/agent_result_v1.py index d661e9cb..94e00d4f 100644 --- a/src/agents_shipgate/schemas/agent_result_v1.py +++ b/src/agents_shipgate/schemas/agent_result_v1.py @@ -30,11 +30,31 @@ ] +#: The JSON-schema default of ``tool.version`` in the published +#: ``docs/agent-result-schema.v3.json`` and +#: ``docs/agent-boundary-result-schema.v3.json``, which both embed this model. +#: The field's default used to be the running package version, so every +#: release rewrote both files' bytes under unchanged schema versions (#1044). +#: Those bytes are now frozen as published: the runtime default is still the +#: running version, and only the schemas keep this constant. A successor schema +#: version must emit no release-specific default. Give it a tool model without +#: ``json_schema_extra`` rather than dropping it here, which would change the +#: bytes of whichever v3 file is still generated from this model. +_PUBLISHED_V3_TOOL_VERSION_DEFAULT = "1.2.0" + + +def _running_version() -> str: + return __version__ + + class AgentResultTool(BaseModel): model_config = ConfigDict(extra="forbid") name: str = "agents-shipgate" - version: str = __version__ + version: str = Field( + default_factory=_running_version, + json_schema_extra={"default": _PUBLISHED_V3_TOOL_VERSION_DEFAULT}, + ) class AgentResultSubject(BaseModel): diff --git a/tests/shard_seconds.json b/tests/shard_seconds.json index 46cb8732..71e5c1aa 100644 --- a/tests/shard_seconds.json +++ b/tests/shard_seconds.json @@ -439,7 +439,8 @@ "tests/test_claude_command_settings.py": 21.0, "tests/test_step_summary_isolation.py": 3.9, "tests/test_host_grants_yaml_timestamps.py": 2.1, - "tests/test_dependency_floors.py": 0.1 + "tests/test_dependency_floors.py": 0.1, + "tests/test_published_schema_freeze.py": 17.1 }, "measured_at": "cbff1ef0b101", "measurement": { @@ -593,6 +594,17 @@ "case_count": 22, "command": "python -m pytest -m \"not slow\" tests/test_dependency_floors.py --junitxml=dependency-floors.xml", "time_basis": "JUnit testcase time; original full-suite measurements retained" + }, + { + "source": "junit", + "measured_at": "4809636c2d with issue #1044 published-schema freeze increment", + "report_sha256": "8e09f135dca74d3f9ffcab16012469d276cb39e9ce4366e538df5cbe6f2f125f", + "files": { + "tests/test_published_schema_freeze.py": 17.1 + }, + "case_count": 13, + "command": "python -m pytest tests/test_published_schema_freeze.py -n 4 --junitxml=freeze-timing.xml", + "time_basis": "JUnit testcase time; original full-suite measurements retained" } ] } diff --git a/tests/test_effect_coverage.py b/tests/test_effect_coverage.py index 4fee7aec..d03416c1 100644 --- a/tests/test_effect_coverage.py +++ b/tests/test_effect_coverage.py @@ -12,7 +12,6 @@ from __future__ import annotations -import hashlib import json from pathlib import Path from typing import cast @@ -873,112 +872,21 @@ def test_new_content_reaches_only_the_current_schema( assert marker in current_text -#: sha256 of each frozen document as published. Checked in rather than read -#: from git history: the main CI job checks out with ``fetch-depth: 1``, so a -#: ``git show `` guard skipped in exactly the runs that must enforce -#: this — a silent skip on the invariant, which is worse than not having it -#: (PR #413 review 3). A deliberate re-freeze updates the hash here, in the -#: same commit, where a reviewer sees it. -_PUBLISHED_SCHEMA_SHA256 = { - "report-schema.v0.35.json": ( - "cd3a971dd6a02676cfa0db798a443715778b364bd80510b23f3c83ea876003cf" - ), - "packet-schema.v0.12.json": ( - "8bef394df037374a14153b743f46a4e279f65b8e1312842efcdec92a07d0bcb5" - ), - "verifier-schema.v0.9.json": ( - "d477db7c202ba9c0629fa28ade7a69d37758052de370aa3246135ede9eeefeaf" - ), - "capability-lock-schema.v0.6.json": ( - "0e43ccadea3258323cac279a59a431e771c2abbe9f4333c853ec5d1401f5285a" - ), - "capability-lock-diff-schema.v0.7.json": ( - "e73210870bb5b1181fcbb90872ced3c71f3ecc1f11124188aa21579694eec93a" - ), - "report-schema.v0.36.json": ( - "c9c0d7576b23fbacfff2da390d462b66add673f448c28eb8295938cf2f986308" - ), - "packet-schema.v0.13.json": ( - "80e6665a4a4d4778b2259c96f756983b8f6857ebca3cb06ecf724c80cfae06eb" - ), - "verifier-schema.v0.10.json": ( - "c582772273cadcb8abd2137b03486528d681093e088a7284e754a8c75f92e727" - ), - "report-schema.v0.37.json": ( - "c5a8551abfab7a95f3c40b2470eb687a2fc95f4c572da318d5a38cfc91933b43" - ), - "packet-schema.v0.14.json": ( - "7bb394413688adcdbbc7f4c599fd361d0c1280c3bf7a049a72d782a7852b85f8" - ), - "verifier-schema.v0.11.json": ( - "adbbcd85f870b8f12f060c02b090395a34ffec4f660f1994d6001a2728397090" - ), - "report-schema.v0.38.json": ( - "df00a7abfbd59fa3274caacd78acf5960f8c754fab2d6e1a62840b7997ca8d24" - ), - "packet-schema.v0.15.json": ( - "b6a47a71482592732c2cac0120a154ce39a6830ed06cc457f6c1fdbb842bdead" - ), - "verifier-schema.v0.12.json": ( - "fae9348ede7e04cd786f2104c55b80b4a9c623c69af1351dc6bca15af2037736" - ), - "capability-lock-schema.v0.7.json": ( - "a90789c318f4a4c19347be9fc440d32322f9befa0c94461d10fb718f5a276704" - ), - "capability-lock-diff-schema.v0.8.json": ( - "c872af71d41a445d1e28065704e034b2a0a32c88a5ec9e767ad8853f4373ac20" - ), - "report-schema.v0.39.json": ( - "9a150c97a13c59b4f1ba68d86d4393be1c799e6c84d1638acee825e28646dab9" - ), - "report-schema.v0.41.json": ( - "2d201cc9767ae8707c1c14dc9a56d49b65429d5d7c32ad6afc30b2c4d66f879b" - ), - "report-schema.v0.40.json": ( - "eee3c02e3bdb25d12c2e3ec46a2c72407f735252f1e6d0804e23c43cf5e86aff" - ), - "packet-schema.v0.16.json": ( - "47c3aee2737985c63899bd110f6805fdcd40172e47251fd215342bc728351e06" - ), - "verifier-schema.v0.13.json": ( - "e692bf97e2880efab8f8d480da453d6094112664e109548cc4f26e48031f00a7" - ), - "report-schema.v0.42.json": ( - "3f149478eb62809d3a40de060269520395ca763496a57ec0bc4a8d37e9b0076e" - ), - "packet-schema.v0.17.json": ( - "269f7752f6cbe5b9097b259d18939ea64a06250e0802cf6ee6b2f655432164e8" - ), - "verifier-schema.v0.15.json": ( - "f362d0bb211830d4900fca2e2f3d38e95256a9e15338ff992a88a03640f29ee5" - ), - "report-schema.v0.43.json": ( - "e3d13aca31480e170e1c6656a4f075191ddc4da747cf00f5c5c7066d266ac7bd" - ), -} - - -def test_every_frozen_schema_has_a_pinned_hash() -> None: - """No frozen document may sit outside the guard.""" +def test_every_frozen_row_is_under_the_byte_freeze() -> None: + """The freeze is about bytes, not only about the token that exposed it. - assert {frozen for frozen, _, _ in _FROZEN_AND_CURRENT_SCHEMAS} == set( - _PUBLISHED_SCHEMA_SHA256 - ) - - -@pytest.mark.parametrize(("frozen", "current", "marker"), _FROZEN_AND_CURRENT_SCHEMAS) -def test_a_frozen_schema_keeps_the_bytes_it_was_published_with( - frozen: str, current: str, marker: str -) -> None: - """The freeze is about bytes, not only about the token that exposed it.""" + The bytes are pinned in one place, ``scripts/published_schema_pins.json``, + over a set derived from the release tags rather than from a hand list, and + ``tests/test_published_schema_freeze.py`` holds every published file to its + pin (#1044). A row above naming an unpinned file would be a frozen document + outside that guard. + """ - del current, marker - digest = hashlib.sha256((_DOCS / frozen).read_bytes()).hexdigest() + pins = json.loads( + (_REPO_ROOT / "scripts" / "published_schema_pins.json").read_text(encoding="utf-8") + )["sha256"] - assert digest == _PUBLISHED_SCHEMA_SHA256[frozen], ( - f"docs/{frozen} is published and pinned; consumers validate against " - "these exact bytes. Emit new content under a new version instead." - ) + assert {frozen for frozen, _, _ in _FROZEN_AND_CURRENT_SCHEMAS} <= set(pins) def test_every_emitted_artifact_validates_against_its_own_current_schema(tmp_path) -> None: diff --git a/tests/test_human_review_request.py b/tests/test_human_review_request.py index 3ade11c1..ff3ef5fc 100644 --- a/tests/test_human_review_request.py +++ b/tests/test_human_review_request.py @@ -187,57 +187,36 @@ def test_runtime_and_discovery_name_the_same_request_contract(): # The schema identifiers below are already published. #536 must not silently # widen any of them when adding a postcondition to a new, separate artifact. # Successor grammars get successor filenames; historical bytes stay readable. -# #561 adds only explanatory metadata to the existing open EvidenceGap. Its -# two additive nodes are checked separately below, then removed to compare -# the entire preceding grammar against its ORIGINAL digest. No control field, -# constraint, or other schema content is exempted from the freeze. -FROZEN_CONTROL_SCHEMAS = { - "verifier-schema.v0.16.json": "cfa834d9bf047d3e39ffed531f19fbd7ed2cd6e82353789dddb7a458dfae408a", - "agent-handoff-schema.v8.json": "036dc757914a297b34ebb9b7ca10c21869c5c91820fa7f07bda415c1effe30c3", - "preflight-schema.v0.4.json": "048c4785253afa476e7b86f6175d61328ea299641f6253c3d4315c9d61d67583", - # Frozen by #610: 0.6 replaced the shared union in preflight's control with - # its own, so 0.5 is a predecessor grammar from here on. - "preflight-schema.v0.5.json": "b85dfd1d1ba84fadb87c739c147ec49d2d329f405f0f0cd2140c1c35667a7947", - "agent-result-schema.v3.json": "ad762ecbbcde20b6cbc117337b4e0ad208708ab062ccc851ffe26b4ff63df232", - "agent-boundary-result-schema.v2.json": "179f849080fabdc59cdf4b86b5ec6a0d9c605e1ac31eb130ee2463c6a0ab361d", - "verify-run-schema.v5.json": "19deb3ba50d3f610325b6e7457ad000ccbcbb737e03cb8bb0e8011c69343f141", - # #662 widened this grammar deliberately, by owner decision: an optional, - # bounded ``capability_rows`` block, published as contract v38 with a - # migration note. The pin moves with that decision and nothing else. - "agent-control-schema.v1.json": "4f28e47bc37deffe2bf9103d9c28bc92809f332dd914c778df6308cfc2dd5e71", -} - - -@pytest.mark.parametrize("filename,digest", FROZEN_CONTROL_SCHEMAS.items()) -def test_review_postcondition_does_not_widen_a_published_control_grammar(filename, digest): +# +# Their bytes are pinned in the one manifest every published schema uses, +# scripts/published_schema_pins.json (#1044). This table used to carry its own +# digests, two of them over bytes no release shipped (verifier 0.16 before +# #561's two open-grammar nodes; agent-result v3 with a 0.16.0 version +# default), so the test undid those changes before hashing. v1.2.0 published +# both files exactly as pinned now, so neither exception is needed (v1.0.0 and +# v1.1.0 published the same verifier 0.16, but each stamped its own version +# into agent-result v3's `tool.version` default, the rewrite #1044 ends). +# #610 froze preflight 0.5 when 0.6 replaced the shared union in +# preflight's control; #662 widened agent-control v1 by owner decision +# (contract v38). +FROZEN_CONTROL_SCHEMAS = ( + "verifier-schema.v0.16.json", + "agent-handoff-schema.v8.json", + "preflight-schema.v0.4.json", + "preflight-schema.v0.5.json", + "agent-result-schema.v3.json", + "agent-boundary-result-schema.v2.json", + "verify-run-schema.v5.json", + "agent-control-schema.v1.json", +) + + +@pytest.mark.parametrize("filename", FROZEN_CONTROL_SCHEMAS) +def test_review_postcondition_does_not_widen_a_published_control_grammar(filename): + pins = json.loads(Path("scripts/published_schema_pins.json").read_text(encoding="utf-8")) raw = (Path("docs") / filename).read_bytes() + assert hashlib.sha256(raw).hexdigest() == pins["sha256"][filename] schema = json.loads(raw) - if filename == "verifier-schema.v0.16.json": - original = copy.deepcopy(schema) - gap = original["$defs"]["EvidenceGap"] - assert gap.get("additionalProperties", True) is True - assert "recovery" not in gap.get("required", []) - assert gap["properties"].pop("recovery") == { - "anyOf": [{"$ref": "#/$defs/CoverageRecovery"}, {"type": "null"}], - "default": None, - } - from agents_shipgate.schemas.coverage_recovery import CoverageRecovery - - assert original["$defs"].pop("CoverageRecovery") == CoverageRecovery.model_json_schema() - raw = (json.dumps(original, indent=2, sort_keys=True) + "\n").encode() - if filename == "agent-result-schema.v3.json": - # The generator stamps the running package version as the default of - # `AgentResultTool.version`, so a release bump moves that one default. - # Only that default is restored to the stamp the digest was taken over; - # the rest of the grammar is still compared byte for byte. - from agents_shipgate import __version__ - - original = copy.deepcopy(schema) - stamp = original["$defs"]["AgentResultTool"]["properties"]["version"] - assert stamp["default"] == __version__ - stamp["default"] = "0.16.0" - raw = (json.dumps(original, indent=2, sort_keys=True) + "\n").encode() - assert hashlib.sha256(raw).hexdigest() == digest action = {"actor": "human", "kind": "review", "command": None, "expects": None, "why": "Review the exact scope"} action_schema = {"$ref": "#/$defs/HumanControlAction", "$defs": schema["$defs"]} diff --git a/tests/test_published_schema_freeze.py b/tests/test_published_schema_freeze.py new file mode 100644 index 00000000..7b93c428 --- /dev/null +++ b/tests/test_published_schema_freeze.py @@ -0,0 +1,408 @@ +"""A published schema document keeps its bytes forever (#1044). + +A ``docs/*-schema*.json`` file that a release tag carries is published: its +``$id`` is a raw URL consumers fetch and pin by digest. ``scripts/ +generate_schemas.py`` holds the whole mechanism -- the pin manifest +(``scripts/published_schema_pins.json``), the published set derived from the +release tags, and the generator's refusal to rewrite a published file. These +tests run it against this tree, and show each guard failing on the defect it +exists for. + +The release tags are required, not optional: a freeze that skipped without +them would skip in exactly the runs that must enforce it (PR #413 review 3). +CI's ``suite`` and ``test`` jobs check out with ``fetch-tags: true``. +""" + +from __future__ import annotations + +import hashlib +import importlib.util +import json +import os +import subprocess +import sys +from pathlib import Path + +import pytest + +from agents_shipgate import __version__ + +REPO_ROOT = Path(__file__).resolve().parent.parent +GENERATOR_PATH = REPO_ROOT / "scripts" / "generate_schemas.py" + +#: The documents whose ``tool.version`` default was the running package +#: version, so that every release rewrote them under an unchanged schema +#: version (#847, #907). +_VERSION_STAMPED = ( + "agent-handoff-schema.v9.json", + "agent-result-schema.v3.json", + "agent-boundary-result-schema.v3.json", +) + +#: The two documents #1088 changed after v1.2.0 published them, and whose +#: current bytes the pin manifest records as a deliberate re-freeze. +_REFROZEN_AFTER_1_2_0 = { + "agent-boundary-result-schema.v3.json", + "verifier-schema.v0.21.json", +} + + +def _load_generator(): + """Import the generator without putting ``scripts/`` on ``sys.path``.""" + + spec = importlib.util.spec_from_file_location( + "agents_shipgate_freeze_generator", GENERATOR_PATH + ) + assert spec is not None and spec.loader is not None + module = importlib.util.module_from_spec(spec) + spec.loader.exec_module(module) + return module + + +@pytest.fixture(scope="module") +def generator(): + return _load_generator() + + +@pytest.fixture(scope="module") +def release_tags(generator): + tags = generator.read_release_tags() + assert tags is not None, ( + "No release tag can be read, so the published set cannot be derived. " + "Fetch them (`git fetch --tags`; in CI, `fetch-tags: true` on the " + "checkout) rather than skipping the freeze." + ) + return tags + + +@pytest.fixture(scope="module") +def released(release_tags) -> str: + """The newest released version, so the controls below hold during release prep too.""" + + return release_tags.names[-1][1:] + + +@pytest.fixture(scope="module") +def generated(generator) -> dict[str, str]: + """Every generator output at this tree's package version, by basename.""" + + return {target.name: content for target, content in (b() for _, b in generator.BUILDERS)} + + +@pytest.fixture() +def docs_copy(generator, tmp_path) -> Path: + """The schema documents, as links into this tree, to break one at a time.""" + + docs = tmp_path / "docs" + docs.mkdir() + for path in (REPO_ROOT / "docs").glob(generator.SCHEMA_DOCUMENT_GLOB): + (docs / path.name).symlink_to(path) + return docs + + +def _kinds(violations) -> dict[str, set[str]]: + found: dict[str, set[str]] = {} + for violation in violations: + found.setdefault(violation.name, set()).add(violation.kind) + return found + + +# --- This tree -------------------------------------------------------------- + + +def test_every_published_schema_document_keeps_its_bytes(generator, release_tags): + violations = generator.freeze_violations(tags=release_tags, version=__version__) + + assert not violations, "\n".join(violation.message for violation in violations) + + +def test_the_published_set_is_every_schema_document_a_release_tag_carries( + generator, release_tags +): + """Derived from the tags, not from a hand list; every one is pinned.""" + + pins = generator.load_schema_pins() + on_disk = {path.name for path in (REPO_ROOT / "docs").glob("*-schema*.json")} + + assert set(release_tags.schema_blobs) <= on_disk + assert set(release_tags.schema_blobs) <= set(pins.sha256) + assert set(_VERSION_STAMPED) <= set(release_tags.schema_blobs) + # The two hand tables this replaces covered 33 of these files. + assert len(release_tags.schema_blobs) >= 177 + + +# --- Each guard fails on its defect ----------------------------------------- + + +def test_a_changed_published_file_fails(generator, release_tags, released, docs_copy): + target = docs_copy / "agent-handoff-schema.v9.json" + text = target.read_text(encoding="utf-8") + target.unlink() + # The exact rewrite a release bump used to make. + assert '"default": "1.2.0"' in text + target.write_text(text.replace('"default": "1.2.0"', '"default": "1.3.0"'), encoding="utf-8") + + violations = generator.freeze_violations(docs=docs_copy, tags=release_tags, version=released) + + assert _kinds(violations) == { + "agent-handoff-schema.v9.json": {"changed", "differs_from_release"} + } + assert "successor schema version" in violations[0].message + + +def test_a_published_file_without_a_pin_fails(generator, release_tags, released, docs_copy): + pins = generator.load_schema_pins() + del pins.sha256["report-schema.v0.43.json"] + + violations = generator.freeze_violations( + docs=docs_copy, pins=pins, tags=release_tags, version=released + ) + + assert _kinds(violations) == {"report-schema.v0.43.json": {"unpinned"}} + assert "--pin" in violations[0].message + + +def test_a_deleted_published_file_fails(generator, release_tags, released, docs_copy): + (docs_copy / "verifier-schema.v0.16.json").unlink() + + violations = generator.freeze_violations(docs=docs_copy, tags=release_tags, version=released) + + assert _kinds(violations) == {"verifier-schema.v0.16.json": {"missing"}} + + +def test_bytes_that_differ_from_a_release_need_a_recorded_refreeze( + generator, release_tags, released, docs_copy +): + """Re-pinning changed bytes is not enough: the release's bytes are the truth. + + Without the manifest's record of #1088, exactly the two files that change + touched fail, so every other published file still has the bytes the newest + release carrying it published. Once a release after 1.2.0 publishes both as + they are, the record is history and removing it changes nothing. + """ + + pins = generator.load_schema_pins() + live = {name for name, tag in pins.refrozen.items() if release_tags.newest(name)[0] == tag} + + violations = generator.freeze_violations( + docs=docs_copy, pins=pins._replace(refrozen={}), tags=release_tags, version=released + ) + + assert _kinds(violations) == {name: {"differs_from_release"} for name in live} + if release_tags.names[-1] == "v1.2.0": + assert live == _REFROZEN_AFTER_1_2_0 + + +def test_while_the_version_has_no_tag_every_schema_document_must_be_pinned( + generator, release_tags, released, docs_copy +): + """Release preparation: the tag will publish every schema document on disk.""" + + (docs_copy / "brand-new-schema.v1.json").write_text("{}\n", encoding="utf-8") + pins = generator.load_schema_pins() + unpinned_on_disk = { + path.name for path in docs_copy.glob("*-schema*.json") if path.name not in pins.sha256 + } + assert "brand-new-schema.v1.json" in unpinned_on_disk + + tagged = generator.freeze_violations(docs=docs_copy, tags=release_tags, version=released) + pending = generator.freeze_violations(docs=docs_copy, tags=release_tags, version="99.0.0") + + assert release_tags.pending(released) is None + assert not tagged + assert _kinds(pending) == {name: {"unpinned"} for name in unpinned_on_disk} + assert all("about to be published at v99.0.0" in violation.message for violation in pending) + + +# --- The generator never rewrites a published file -------------------------- + + +def _with_handoff_builder(generator, *, bumped: bool): + """Two builders: the published agent-handoff v9 and the unpublished checks.json. + + The v9 builder emits the committed bytes, or them with the rewrite every + release bump used to make, so the controls outlive v9's own builder. + """ + + name = "agent-handoff-schema.v9.json" + committed = (REPO_ROOT / "docs" / name).read_text(encoding="utf-8") + assert '"default": "1.2.0"' in committed + content = committed.replace('"default": "1.2.0"', '"default": "1.3.0"') if bumped else committed + + def build() -> tuple[Path, str]: + return generator.DOCS / name, content + + return (("checks_catalog", dict(generator.BUILDERS)["checks_catalog"]), ("agent_handoff", build)) + + +def test_the_generator_writes_nothing_when_a_published_file_would_change( + generator, monkeypatch, tmp_path, capsys +): + monkeypatch.setattr(generator, "DOCS", tmp_path) + monkeypatch.setattr(generator, "BUILDERS", _with_handoff_builder(generator, bumped=True)) + + assert generator.main([]) == 1 + err = capsys.readouterr().err + + assert "docs/agent-handoff-schema.v9.json: published at v" in err + assert "successor schema version" in err + assert "constraints/dev.txt" in err + assert "Nothing was written." in err + # Not even the unpublished checks.json, which an unrefused run writes. + assert list(tmp_path.iterdir()) == [] + + assert generator.main(["--check"]) == 1 + assert "docs/agent-handoff-schema.v9.json: published at v" in capsys.readouterr().err + + +def test_the_generator_writes_when_no_published_file_would_change( + generator, monkeypatch, tmp_path +): + """The positive control: the same run without the change writes both files.""" + + monkeypatch.setattr(generator, "DOCS", tmp_path) + monkeypatch.setattr(generator, "BUILDERS", _with_handoff_builder(generator, bumped=False)) + + assert generator.main([]) == 0 + + written = (tmp_path / "agent-handoff-schema.v9.json").read_bytes() + assert (tmp_path / "checks.json").is_file() + assert hashlib.sha256(written).hexdigest() == ( + generator.load_schema_pins().sha256["agent-handoff-schema.v9.json"] + ) + + +def test_without_release_tags_the_pins_still_refuse_a_rewrite( + generator, monkeypatch, tmp_path, capsys +): + """A shallow clone falls back to the pin manifest instead of failing closed.""" + + monkeypatch.setattr(generator, "read_release_tags", lambda repo_root=None: None) + monkeypatch.setattr(generator, "DOCS", tmp_path) + monkeypatch.setattr(generator, "BUILDERS", _with_handoff_builder(generator, bumped=True)) + + assert generator.main([]) == 1 + err = capsys.readouterr().err + + assert "no release tag can be read here" in err + assert ( + "docs/agent-handoff-schema.v9.json: pinned in scripts/published_schema_pins.json" in err + ) + assert list(tmp_path.iterdir()) == [] + + +def test_pin_adds_the_missing_pin_and_never_moves_one(generator, monkeypatch, tmp_path, capsys): + original = generator.PINS_PATH.read_bytes() + manifest = tmp_path / "published_schema_pins.json" + payload = json.loads(original) + del payload["sha256"]["report-schema.v0.43.json"] + manifest.write_text(json.dumps(payload), encoding="utf-8") + monkeypatch.setattr(generator, "PINS_PATH", manifest) + monkeypatch.setattr(generator, "BUILDERS", ()) + + assert generator.main(["--pin"]) == 0 + assert "Pinned docs/report-schema.v0.43.json" in capsys.readouterr().out + # The seeded manifest is exactly what --pin derives. + assert manifest.read_bytes() == original + + payload["sha256"]["verifier-schema.v0.16.json"] = "0" * 64 + manifest.write_text(json.dumps(payload), encoding="utf-8") + before = manifest.read_bytes() + + assert generator.main(["--pin"]) == 1 + err = capsys.readouterr().err + assert "docs/verifier-schema.v0.16.json: published at v" in err + assert "Nothing was pinned." in err + assert manifest.read_bytes() == before + + +# --- A package version bump leaves every schema document byte-identical ----- + +_BUMP_PROBE = """ +import importlib.util, json, sys + +import agents_shipgate + +agents_shipgate.__version__ = sys.argv[2] +spec = importlib.util.spec_from_file_location("bumped_generator", sys.argv[1]) +generator = importlib.util.module_from_spec(spec) +spec.loader.exec_module(generator) + +from agents_shipgate.schemas.agent_handoff import AgentHandoffTool +from agents_shipgate.schemas.agent_result_v1 import AgentResultTool + +outputs = [builder() for _name, builder in generator.BUILDERS] +refusals = generator._frozen_rewrites( + outputs, pins=generator.load_schema_pins(), tags=generator.read_release_tags() +) +print(json.dumps({ + "runtime": [AgentHandoffTool().version, AgentResultTool().version], + "outputs": {target.name: content for target, content in outputs}, + "refusals": sorted(refusals), +})) +""" + + +def test_a_version_bump_leaves_every_schema_document_byte_identical(generator, generated): + """Simulate the release bump: the same generator under another version. + + Before #1044 this changed the three version-stamped documents, so every + release-prep commit rewrote published bytes under unchanged versions. + """ + + path = [str(REPO_ROOT / "src"), *filter(None, [os.environ.get("PYTHONPATH")])] + env = {**os.environ, "PYTHONPATH": os.pathsep.join(path)} + completed = subprocess.run( + [sys.executable, "-c", _BUMP_PROBE, str(GENERATOR_PATH), "9.9.9"], + capture_output=True, + text=True, + check=False, + cwd=REPO_ROOT, + env=env, + ) + assert completed.returncode == 0, completed.stderr + bumped = json.loads(completed.stdout) + + # The bump took effect: the runtime default still follows the running + # version, and a generated file that does record it moved. + assert bumped["runtime"] == ["9.9.9", "9.9.9"] + assert bumped["outputs"]["determinism-boundary.json"] != generated["determinism-boundary.json"] + + schemas = {name for name in generated if generator.is_schema_document(name)} + assert {name for name in bumped["outputs"] if generator.is_schema_document(name)} == schemas + changed = sorted(name for name in schemas if bumped["outputs"][name] != generated[name]) + assert changed == [] + assert bumped["refusals"] == [] + + +def test_no_unpublished_schema_carries_a_release_specific_default( + generator, release_tags, generated +): + """A successor schema version must not inherit the frozen version default. + + The three version-stamped documents keep the ``"default": "1.2.0"`` that + v1.2.0 published. Any schema document not yet published that carries + a release version as a default is the #1044 defect starting again. + """ + + releases = {tag[1:] for tag in release_tags.names} | {__version__} + published = set(release_tags.schema_blobs) | set(generator.load_schema_pins().sha256) + + def release_defaults(node) -> list[str]: + if isinstance(node, dict): + default = node.get("default") + found = [default] if isinstance(default, str) and default in releases else [] + return found + [value for child in node.values() for value in release_defaults(child)] + if isinstance(node, list): + return [value for child in node for value in release_defaults(child)] + return [] + + offenders = { + name: release_defaults(json.loads(content)) + for name, content in generated.items() + if generator.is_schema_document(name) and name not in published + } + + assert {name: found for name, found in offenders.items() if found} == {} + for name in set(_VERSION_STAMPED) & set(generated): + assert release_defaults(json.loads(generated[name])) == ["1.2.0"]