Skip to content

Refactor tests around one entry and explicit data ownership - #69

Merged
ZSeanYves merged 3 commits into
mainfrom
codex/tests-structure
Nov 2, 2026
Merged

ZSeanYves merged 3 commits into
mainfrom
codex/tests-structure

Conversation

@ZSeanYves

@ZSeanYves ZSeanYves commented Nov 1, 2026 •

Copy link
Copy Markdown
Collaborator

Test automation mixed CLI cases, release selections, coverage reviews, generated evidence and command-specific assertions across overlapping directories. This change establishes tests/run.mbtx as the local/CI entry, gives current data explicit owners, and removes the old execution paths without forwarding wrappers.

  • Separate generic harness/driver/catalog code, specialized scenarios, current cases, release policy, reviewed coverage and generated definitions. Split network, policy and resource code by responsibility.
  • Add index check/update; CI rejects stale generated quality, coverage, gap and candidate files. Committed gap indexes describe definitions only; actual execution receipts remain under fresh report roots. Scope LF checkout rules to these JSON definitions so byte comparisons remain deterministic on Windows.
  • Check and format every standalone script. Remove case overlays, duplicate current-case manifests, unused backend parameters, retired CLI aliases and published-only legacy branches.
  • Preserve all current per-command case files and static corpus bytes, and move published latest.json/latest-cases.json byte-for-byte. Product source, APIs and package versions are unchanged.

Validation on macOS: 1,800 Wasm package tests; 201 Native tool tests; 2,274 functional passes and 21 explicit platform skips; all 652 reviewed coverage obligations qualified, with 213 unresolved obligations still reported. Unified product/tool/script checks, generated-index check, explicit formatting, architecture validation (80 commands/127 packages), nested-directory selection, published-manifest validation and local documentation links pass.

Required Linux, macOS and Windows development checks pass at 732085a6c714529628240691f58f89f1792c18d7, including pinned Linux upstream comparisons. The first Windows attempt hit four existing 10-second PowerShell observer timeouts in mv link tests; one Windows-only rerun passed all 1,800 package tests, 201 tool tests and functional checks with the same assertions and timeout settings. The original failure remains recorded.

Manual resource/performance and uploaded-coordinate acceptance workflows were not run. Actual-main validation remains after merge.

@ZSeanYves
ZSeanYves marked this pull request as ready for review November 1, 2026 13:37
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 1, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-01T13:39:37.400637Z 732085a Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 732085a6c7

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

"candidate": true,
"core_version": module_field("core/moon.mod", "version"),
"base_manifest": "active-cases.json",
"base_manifest": "../cases/manifest.json",

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Allow the candidate's cross-directory case reference

When the release-acceptance workflow selects candidate, this generated base_manifest is rejected before any acceptance cases run: published_coordinates() in tests/harness/driver/published.mbt passes it to safe_relative_path(), which explicitly rejects every .. component. Consequently, both the committed candidate and every candidate regenerated by index update fail with unsafe published case path, although the default latest manifest still works. Update the manifest-path validation to support the new cross-directory reference while retaining an appropriate containment check.

Useful? React with 👍 / 👎.

@ZSeanYves
ZSeanYves merged commit 9814ad7 into main Nov 2, 2026
5 of 6 checks passed
@ZSeanYves
ZSeanYves deleted the codex/tests-structure branch November 2, 2026 21:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant