Skip to content

PPF-2106: Validate the exact release revision and artifact before publication - #2109

Merged
chinapandaman merged 3 commits into
masterfrom
PPF-2106
Nov 7, 2026
Merged

chinapandaman merged 3 commits into
masterfrom
PPF-2106

Conversation

@chinapandaman

@chinapandaman chinapandaman commented Nov 7, 2026 •

Copy link
Copy Markdown
Owner

Resolves #2106 (action plan item E3-08, review finding R13).

Baseline

v6.1.0 was built and uploaded from 1b3b5d5b, the [skip ci] version bump commit. Tests and Linting never ran on it. The Deploy workflow ran uv build && uv publish with no tests, no linting, and no install of the built files. Create Release ran no checks before tagging.

Changes

  • Reusable gates: Tests and Linting gain a workflow_call trigger. Tests gains a distribution job that builds the sdist and wheel once with uv build and checks them with scripts/check_dist.py on Python 3.11 and 3.15. It then uploads them as the dist artifact and records the commit and SHA-256 checksums in the run summary and as a workflow output. The job runs on every PR and push.
  • Deploy (python-publish.yml): runs Tests and Linting on the release commit. The upload job needs both and is the only job with id-token: write. It downloads dist, fails unless the files' checksums exactly match the recorded ones, and uploads those files. The trusted-publisher workflow file is unchanged.
  • Dry run: Deploy also has workflow_dispatch. A manual run runs every check, including downloading dist and comparing its checksums in the upload job. Only the PyPI upload and its summary step carry if: github.event_name == 'release'. Job permissions can't depend on the event, so the upload job also holds id-token: write during a dry run, but no step that runs then publishes.
  • Create Release: runs Tests and Linting on the dispatched commit before the release job. Only the release job keeps contents: write.
  • scripts/check_dist.py:
    • Version: the file names, sdist and wheel metadata, installed __version__, and release tag (on release events) must all agree.
    • Contents: the wheel must hold exactly the PyPDFForm/** files tracked at HEAD, compared by git blob ID, plus .dist-info. The sdist must hold the same package files plus README.md, LICENSE, and pyproject.toml. Beyond those, it may contain only generated metadata and tracked top-level tests/test*.py files, which matches its current contents.
    • Install: for each Python, the wheel is installed into new venvs outside the checkout, once without extras and once with [cli]. A copy of the script runs there with python -I and PYTHON* variables removed. It requires PyPDFForm to load from that venv's site-packages and compares the installed files with the wheel. It imports every module (CLI modules only with the extra) and creates, fills, writes, and reopens a text/checkbox/dropdown form.
    • CLI: pypdfform must print install guidance and exit 1 without the extra. With it, --version, create blank, create field, fill, and inspect data must all work.
  • Docs: the release guide covers the gates, dry run, failure recovery, and running the check locally. The pull request requirements mention the new check. The action plan records E3-08 in a separate commit.

Maintainer decisions: gate in Deploy and also before tagging in Create Release; build and check inside the Tests workflow; the smoke test as a script in scripts/; enforce the current sdist contents; add the guarded dry run. The partial tests/ in the sdist is left for a separate packaging issue. Deploy Docs is unchanged.

Validation

  • Harness: kept outside the repository, it ran the real script against builds of disposable clones. All 63 cases passed:

    • clean builds, including with PYTHONPATH and VIRTUAL_ENV pointing at the checkout
    • wrong tags and dist-directory shapes
    • 9 tampered wheels and 15 tampered sdists
    • 12 broken commits, ten of which build archives that still match HEAD and fail only in the install checks
    • an editable checkout install, which the probe rejected
    • missing tools, a non-git source, and uncommitted or untracked package files

    Every failure exited 1 with a message and no traceback from the script itself.

  • Workflows: actionlint 1.7.12 with ShellCheck 0.11.0 passed the four changed workflows. It also caught a misspelled output name planted in a copy, so it checks the cross-workflow wiring.

  • Checksum step: the Deploy verification script, taken from the YAML and run with bash -e against the real dist artifact from this PR's CI, passed with the recorded checksums. It failed for a tampered wheel, an extra file, a missing sdist, empty checksums, and checksums of other files. It also failed when sha256sum was unavailable.

  • CI: the distribution job passed on this PR. It checked the sdist (84 files) and wheel (58 package files) of the PR's merge commit, and installed the wheel on Python 3.11.18 and 3.15.2, without extras and with [cli].

  • Checks: make test-all (468 passed, 100% coverage), make linting, ruff format --check, git diff --check, and an mkdocs build all passed, using local ./venv with standard zlib.

Not yet exercised

  • PR CI runs only the distribution job of the new gate. The workflow_call paths, the artifact hand-off, and the checksum comparison first run after merge; a Deploy dry run on master before the next release would cover them. The tag check runs only in a real release.
  • If Deploy's checks fail, the GitHub release, its tag, and the docs already exist without a PyPI upload. Create Release's pre-check makes that unlikely.
  • distribution is not a required status check until it is added to branch protection.
  • The contents check compares git blob IDs, so a Windows checkout that converts line endings will report false mismatches locally. CI runs on Linux.
  • The development container was not used.

🤖 Generated with Claude Code

chinapandaman and others added 2 commits November 7, 2026 08:42
…lication

- Let other workflows call Tests and Linting. Add a `distribution` job
  to Tests that builds the sdist and wheel once, checks them with the
  new scripts/check_dist.py on Python 3.11 and 3.15, uploads them as
  the `dist` artifact, and records the commit and SHA-256 checksums.
- Deploy runs Tests and Linting on the release commit. Its upload job,
  the only one with id-token: write, downloads `dist`, fails unless the
  checksums match the recorded ones, and uploads exactly those files.
  A manual run is a dry run that skips the upload.
- Create Release runs Tests and Linting on the dispatched commit before
  creating the tag or release.
- check_dist.py checks that the version agrees across file names,
  metadata, `__version__`, and the release tag, and that the sdist and
  wheel contents match HEAD byte for byte. It then installs the wheel
  outside the checkout, without extras and with the cli extra, and
  checks it there with `python -I`: imports come from site-packages,
  every module imports, a form is created, filled, and reopened, and
  the `pypdfform` command works.
- Describe the gates, dry run, recovery, and local check in the
  release guide and pull request requirements.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Mark E3-08 done and record the baseline, implementation, confirmed
  policy, validation, and remaining limitations.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@chinapandaman chinapandaman self-assigned this Nov 7, 2026
@chinapandaman chinapandaman added the enhancement New feature or request label Nov 7, 2026
@chinapandaman chinapandaman moved this from Backlog to In review in PyPDFForm-support-2026 Nov 7, 2026
@chinapandaman

Copy link
Copy Markdown
Owner Author

Found one issue at 17a9d471652cceef634d2b39fd4d6fa30e7304b2:

[P2] Keep artifact verification active during dry runs — .github/workflows/python-publish.yml:17–19

The if: github.event_name == 'release' condition skips the entire deploy job during a manual workflow_dispatch, including artifact download and checksum verification. A rehearsal can therefore pass despite broken artifact handoff or checksum output wiring, although the release guide says it runs every check and skips the upload. Please keep download and checksum verification active during manual runs and guard only publication and its success summary.

Validation: built the PR in a disposable checkout and ran the distribution smoke checks on Python 3.14 with and without [cli]; both passed. Confirmed that changed package bytes, an extra distribution file, and a wrong release tag are rejected. Ruff lint/format and git diff --check passed. The full test/lint matrix was assessed through the passing PR CI rather than rerun locally.

- Run the Deploy upload job on every event, so a manual dry run also
  downloads the checked distribution and compares its checksums. Only
  the PyPI upload and its summary are limited to release events.
- Say so in the release guide and the E3-08 action plan entry, and
  note that the upload job holds id-token: write during a dry run.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@chinapandaman

Copy link
Copy Markdown
Owner Author

Fixed the P2 dry-run finding in 2bbf12c. The deploy job no longer has a job-level if, so a manual run downloads dist and compares its checksums too. Only "Deploy to PyPI" and "Record the published commit and files" carry if: github.event_name == 'release'. The release guide and the E3-08 entry now say this.

One trade-off: job permissions can't be conditional, so the job also holds id-token: write during a dry run, though no step that runs then publishes. Anyone who can dispatch the workflow can already push a branch with a modified workflow, so the added exposure is small. If you'd rather keep the token out of dry runs entirely, I can move verification into a separate read-only job and repeat it in a release-only upload job.

Validation: I ran the verification step's script from the YAML with bash -e against the real dist artifact from this PR's CI run. It passed with the recorded checksums and failed for a tampered wheel, an extra file, a missing sdist, empty checksums, and mismatched checksums. It also failed when sha256sum was missing. actionlint with ShellCheck passed. The dispatch itself can only run once the workflow is on master.

@chinapandaman
chinapandaman merged commit 27d3576 into master Nov 7, 2026
21 checks passed
@chinapandaman
chinapandaman deleted the PPF-2106 branch November 7, 2026 14:59
@github-project-automation github-project-automation Bot moved this from In review to Done in PyPDFForm-support-2026 Nov 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

PPF-2106: Validate the exact release revision and artifact before publication

1 participant