Repository navigation
fix(schemas): freeze published schema bytes; the generator refuses to rewrite one (#1044 slice A) - #1116
Open
pengfei-threemoonslab wants to merge 2 commits into
Open
fix(schemas): freeze published schema bytes; the generator refuses to rewrite one (#1044 slice A)#1116pengfei-threemoonslab wants to merge 2 commits into
pengfei-threemoonslab wants to merge 2 commits into
Conversation
… rewrite one (#1044 slice A) Three published schemas embedded the package version as a JSON-schema default (`tool.version`), so every release-prep bump rewrote docs/agent-handoff-schema.v9.json, docs/agent-result-schema.v3.json and docs/agent-boundary-result-schema.v3.json under unchanged schema versions (#847, #907), against docs/report-1-0-contract.md rule 3. - AgentHandoffTool and AgentResultTool: `version` keeps the running version at runtime (default_factory) while the schema keeps the published "1.2.0" (json_schema_extra). Byte-identical output, proven under a simulated bump; a comment says successors emit no release-specific default. - scripts/generate_schemas.py: one freeze mechanism. The published set is every docs/*-schema*.json a `v<version>` tag carries (from git, not a hand list) plus every pinned file, and, while the package version has no tag, every schema document on disk. When regenerating would change a published or pinned file, the generator writes nothing, exits 1 and names the successor-version route (and the locked-pydantic cause). `--check` also reports a published file that is missing, unpinned, changed, or different from the newest release that carried it. `--pin` adds missing pins and never moves one. Without tags (shallow clone, sdist) the published set falls back to the pin manifest, which CI keeps complete. - scripts/published_schema_pins.json: 177 pins seeded from current bytes, the union of both former tables. agent-boundary-result v3 and verifier 0.21 are recorded as re-frozen relative to v1.2.0 (#1088). - tests: the two hand pin tables (test_effect_coverage _PUBLISHED_SCHEMA_SHA256, test_human_review_request FROZEN_CONTROL_SCHEMAS digests) now read the one manifest. The #561 and version-stamp exceptions are gone: both covered bytes no release shipped, and v1.0.0-v1.2.0 published both files as now pinned. tests/test_published_schema_freeze.py runs the mechanism on this tree and shows each guard failing on its defect. - Docs: report-1-0-contract rule 3, STABILITY, CONTRIBUTING, release-runbook step 3 (pin at the version bump), CHANGELOG. Refs #1044
…fault The test comment said v1.0.0 through v1.2.0 published agent-result v3 exactly as pinned; v1.0.0 and v1.1.0 each stamped its own version into its `tool.version` default, so only v1.2.0 did. STABILITY.md, docs/report-1-0-contract.md and the freeze test's docstring now say the `1.2.0` default is the value v1.2.0 published, not the one these documents were always published with.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Protected paths: yes (
scripts/generate_schemas.py, the newscripts/published_schema_pins.json, the tests that pin published schemas:tests/test_effect_coverage.py,tests/test_human_review_request.py, the newtests/test_published_schema_freeze.py; and the two models that render the frozendocs/agent-handoff-schema.v9.json,docs/agent-result-schema.v3.jsonanddocs/agent-boundary-result-schema.v3.json, whose bytes do not change) — needs owner review before merge.Refs #1044 (slice A only; the issue stays open for slice B, the diff output schemas).
Must land before the v1.3.0 version bump. Until it does, the bump rewrites the three files below again, and after it the bump PR must run
python scripts/generate_schemas.py --pin(the freeze test fails on that PR until it does; see Release-readiness notes).The defect.
version: str = __version__inAgentHandoffToolandAgentResultToolput the package version into the JSON-schema default oftool.version, so every release-prep bump rewrotedocs/agent-handoff-schema.v9.json,docs/agent-result-schema.v3.jsonanddocs/agent-boundary-result-schema.v3.jsonunder unchanged schema versions (#847, #907). That breaksdocs/report-1-0-contract.mdrule 3. The guard was two hand-written pin tables covering 33 of 177 tag-published files. Its completeness test compared one hand list with the other.What changed.
schemas/agent_handoff.py,schemas/agent_result_v1.py):versionisField(default_factory=_running_version, json_schema_extra={"default": "1.2.0"}). At runtime the default is still the running version. The schema keeps the"1.2.0"the files were published with. Generated output is byte-identical:--checkis clean and_canonical_jsonsorts keys. A comment on each says a successor schema version must emit no release-specific default and how to drop it.AgentResultToolfeeds both v3 files, so its comment says to give a successor its own tool model rather than drop the extra in place.scripts/generate_schemas.py:docs/*-schema*.jsonthat av<version>release tag carries, read from git (git ls-tree <tag> -- docs/), not from a hand list. Every pinned file counts too. While the package version has no tag (release preparation), every schema document on disk counts.freeze_violations()reports a published file that is missing, has no pin, no longer matches its pin, or differs from the bytes the newest release carrying it published.--checkruns it, and so does the test.--checkreports the same refusal. Unpublished files keep today's--checkbehaviour.--pinadds pins for published files that have none. It never moves an existing pin, needs tags, and refuses while anything drifts.suiteandtestjobs (and both release-verify workflows) check out withfetch-tags: true(.github/workflows/ci.ymlsuite and test checkouts). There, the freeze test fails if a tag-published file has no pin, so the manifest names every published file. Failing closed would break the generator for every contributor without tags while protecting nothing more. The freeze tests need tags and fail, rather than skip, without them (PR fix(evidence): a declaration cannot discharge a category it does not cover (#409 follow-ups) #413 review 3).scripts/published_schema_pins.json: 177 sha256 pins seeded from current bytes, plus arefrozenrecord of the two build(deps): refresh dev, build-backend and publish locks; uv 0.13.1, pydantic 2.14.1 #1088 files (below). All 25_PUBLISHED_SCHEMA_SHA256digests and 6 of 8FROZEN_CONTROL_SCHEMASdigests carried over unchanged. The other two are explained in the next bullet.tests/test_effect_coverage.pyloses_PUBLISHED_SCHEMA_SHA256and its two tests. It keeps the per-row marker test, and the rows must now be a subset of the manifest.tests/test_human_review_request.py'sFROZEN_CONTROL_SCHEMASbecomes a list of names. The test hashes against the manifest and keeps itsHumanControlActiongrammar assertion.cfa834…) and the agent-result v3 "restore0.16.0" digest (ad762e…) were taken over bytes no release tag ever carried. v1.0.0, v1.1.0 and v1.2.0 all publishedverifier-schema.v0.16.jsonexactly as pinned now (86c674…), and agent-result v3 is pinned at v1.2.0's bytes. Pinning current bytes needs neither exception.docs/report-1-0-contract.mdrule 3 (no longer names_PUBLISHED_SCHEMA_SHA256),STABILITY.mdandCONTRIBUTING.md§ Schema Changes.docs/release-runbook.mdstep 3 says the change that sets the version runs the generator, then--pin. OneCHANGELOG.mdline goes under### Changes, after the empty-preflight entry.Owner decision
1.3.0 will publish
docs/agent-boundary-result-schema.v3.jsonanddocs/verifier-schema.v0.21.jsonwith bytes that differ from what 1.2.0 published. #1088 (pydantic 2.14 lock refresh) added"additionalProperties": falsetoCapabilityDiffRowin both, in place, after v1.2.0. CHANGELOG## Unreleasedrecords it. As the 2026-11-05 review recommends, this PR freezes them at their current main bytes and records that in the manifest'srefrozenblock (differs_from_release: v1.2.0). Without the record,freeze_violationsreports exactly these two files as differing from v1.2.0 (test_bytes_that_differ_from_a_release_need_a_recorded_refreeze). A consumer that pinned the 1.2.0 bytes by sha256 sees a mismatch at 1.3.0 under unchanged schema versions.Alternative: restore the 1.2.0 bytes. Check both files out from
v1.2.0, add a generator post-process that deletes that oneadditionalPropertiesfromCapabilityDiffRowin both, drop the tworefrozenentries, and withdraw the #1088 CHANGELOG line. Costs:Benefit: digest pins taken on 1.2.0 keep matching.
Pydantic is a second way published bytes can drift. The runtime floor is
pydantic>=2.12.5and the dev lock is 2.14.3 (#1107, after #1088's 2.14.1). Thedependency-floorsjob does not rungenerate_schemas.py, on purpose (its comment cites #1088). None of its listed test files runs the freeze tests. I ran the generator at the floor (pydantic 2.12.5) and at 2.13.4. It refuses to rewriteverifier-schema.v0.21.jsonandagent-boundary-result-schema.v3.json, writes nothing, exits 1, and namesconstraints/dev.txt. Before this PR it silently rewrote both published files. This machine's shared.venvhas 2.13.4, so--checkfailed onmainthere for exactly this reason.Type
Authorship
Acceptance criteria
tests/test_published_schema_freeze.py::test_a_version_bump_leaves_every_schema_document_byte_identicalruns every builder in a subprocess withagents_shipgate.__version__ = "9.9.9". It asserts that every*-schema*output equals this version's and that the generator would refuse nothing. It also checks that the bump took effect: both tool models default to9.9.9, anddeterminism-boundary.jsonmoved.__version__ = "1.3.0", no.git(so the pin fallback): write exits 0,--checkexits 0, nodocs/*-schema*.jsondiffers from the worktree, and onlydeterminism-boundary.{json,md}moved.version: str = __version__, the bump test fails on the three files. In the scratch copy, write exits 1 withRefused: regenerating would change 3 published schema file(s). Nothing was written.and--checkexits 1.test_the_generator_writes_nothing_when_a_published_file_would_change: v9 is changed the way a bump used to change it, next to the unpublishedchecks.json. Write mode exits 1, the output directory stays empty, the message namespublished at v1.2.0and the successor route, and--checkreports the same.test_the_generator_writes_when_no_published_file_would_change: the same run without the change writes both files, and v9's bytes equal its pin.test_without_release_tags_the_pins_still_refuse_a_rewrite.test_every_published_schema_document_keeps_its_bytesrunsfreeze_violationson this tree.test_the_published_set_is_every_schema_document_a_release_tag_carrieschecks that the tags carry at least 177 files, all on disk and all pinned.changed+differs_from_release; a dropped pin givesunpinned; a deleted file givesmissing; an emptyrefrozenreports exactly the two build(deps): refresh dev, build-backend and publish locks; uv 0.13.1, pydantic 2.14.1 #1088 files. While the version has no tag, every unpinned schema document must be pinned, including a new one."default": "1.3.0"in v9 makes--checkexit 1 with both problems, andtest_every_published_schema_document_keeps_its_bytesfails. Deleting thereport-schema.v0.43.jsonpin makes--checkexit 1 (has no pin … --pin).--pinthen restores the manifest byte-identical, andtest_pin_adds_the_missing_pin_and_never_moves_oneasserts the same round trip.test_no_unpublished_schema_carries_a_release_specific_defaultcovers it. With_AGENT_RESULT_SUFFIX = "v4"(a successor cut without dropping the frozen default), it fails on{'agent-result-schema.v4.json': ['1.2.0']}.Verification
Local runs used a scratch venv built from
constraints/dev.txtwith--require-hashes(pydantic 2.14.3) andPYTHONPATH=src.agents_shipgate.__file__resolved to this worktree. The shared.venvhas pydantic 2.13.4, older than the lock. The full suite was not run locally; CI's six shards are the gate.python scripts/generate_schemas.py --checkpython scripts/generate_schemas.py --pinEvery published schema document is already pinned.)python -m pytest tests/test_published_schema_freeze.py tests/test_effect_coverage.py tests/test_human_review_request.py tests/test_schema_roundtrip.py tests/test_determinism_boundary.py tests/test_report_1_0_contract.py tests/test_remediation_metadata.py tests/test_reports.py tests/test_public_surface_contract.py tests/test_distribution_surface_parity.py tests/test_shard_partition.py tests/test_release_pipeline.py tests/test_agent_handoff.py tests/test_agent_boundary.py tests/test_codex_boundary_check.py -n 4test_distribution_surface_parity.py)python -m ruff check .--check--check_AGENT_RESULT_SUFFIX = "v4"tests/shard_seconds.jsongains the new file's measured weight (17.1 s at-n 4) and a supplemental measurement record, as #1092 and #1105 did.Release-readiness notes
No user-code import added to default scan paths
No network access added to default scan paths
New or changed check IDs are documented in
docs/checks.md(none changed)Report/schema changes are additive or documented in
STABILITY.md(no schema bytes change; the freeze rule is documented)Lands before the version bump. The bump PR must run
python scripts/generate_schemas.py --pin, which pins everydocs/*-schema*.jsonthe tag will publish. Today that is the nine files new since 1.2.0 (host-grants inventory/baseline/drift v0.8 and v0.9, openshell-native evidence/trust v1, preflight v0.6), plus any this batch adds, such as slice B's. While__version__has no tag,test_every_published_schema_document_keeps_its_bytesfails until they are pinned, so nothing reaches the tag unpinned.Coordination. After this lands, a PR that edits a tag-published schema in place fails
--checkand the generator refuses it. A schema that is new since 1.2.0 can still change freely until the bump pins it.Runtime behaviour does not change:
tool.versionin emitted handoff, agent-result, agent-boundary and codex-boundary artifacts is still the running version.Out of scope
docs/manifest-v0.1.json(an input grammar, extended in place up to An MCP server needs 116 per-tool selectors to bind its own published surface; the two shorter spellings both dead-end #432) and other non-*-schema*JSON underdocs/. They are outside thedocs/*-schema*.jsonfreeze set the issue names.AGENTS.md, skills and plugins: another PR in this batch edits the agent-facing copies.docs/agent-contract-current.mdstill says "every published schema URL keeps its bytes forever", which stays true.🤖 Generated with Claude Code