Skip to content

Keep output deterministic and add quiet and verbose controls - #392

Open
saioai wants to merge 28 commits into
mainfrom
codex/universal-output-20261008
Open

saioai wants to merge 28 commits into
mainfrom
codex/universal-output-20261008

Conversation

@saioai

@saioai saioai commented Oct 8, 2026 •

Copy link
Copy Markdown
Contributor

Forced color could put ANSI escapes into piped JSON. Structured streams could also wait for another event before printing.
This removes added color from piped results and emits stream events promptly.
Quiet and verbose controls preserve selected data and failure status.

In this standalone PR, binary-save and manpage receipts still reach stdout and ignore quiet.
The API binary-receipt fix is published in #395; composition and combined validation remain pending.
@manpages still needs the separate generator correction, whose publication remains blocked.
The recorded native Windows lifecycle checks also remain open.

What changes

  • Pipes and files receive no added ANSI decoration, including forced color. Terminal JSON also respects NO_COLOR.
  • Noninteractive structured streams emit each event immediately. Early exits close owned SDK streams and preserve cleanup failures.
  • Quiet suppresses optional hints and image progress. Saved image paths, selected API events, and explicitly requested final previews remain.
  • Verbose reports command details, elapsed time, and the final result on stderr, including request-setup failures.
  • Quiet, machine errors, and interruptions suppress optional verbose details. Command and cleanup failures retain their causes and exit codes.
  • Human API errors include accepted returned request IDs and concise recovery guidance. Structured error bytes remain unchanged.
  • Existing formats, extraction, list records, limits, pagination, raw strings, and binary bytes retain their contracts.

Commands

The new flags are --quiet and --verbose, grouped under Output in root help.
Quiet overrides verbose. -v still means version.

openai files list --format jsonl --quiet > files.jsonl
openai models list --verbose                    # details on stderr
openai models list --format jsonl --verbose --format-error text > models.jsonl
openai responses create --model MODEL --input "Hello" --stream=true --format jsonl

The default remains readable text, including pipes. A destination filename does not select a format.
JSON lists still emit independent values. This change introduces no arrays, JSON/raw aliases, or CI output mode.
Explicit explore retains terminal interaction in CI. Quiet does not disable requested interaction.
--debug remains a separate troubleshooting control.

Code

pkg/custom owns feedback policy, human errors, and image-progress orchestration.
main.go wraps the existing request-configuration runner so verbose reporting includes setup and cleanup failures.
Existing formatters and iterators retain data selection and error ownership. Transformers remain unchanged.
This adds no package or dependency and does not change generated sources.

Tested

Candidate: 61e8db6 against main 6fcc478.
Runtime remains identical to the tested af738605; the last two commits correct validation assets.
Independent adversarial review passed 121 executable checks and three clean-binary controls.
Public regression checks passed 83 groups. Lifecycle race checks, required Bash/zsh checks, and selected large-payload cases passed.
Clean and single-file builds, focused vet, and module verification passed for the unchanged runtime.
The trusted hosted budget check passes at 21/1000 lines.

Six native image probes and five terminal regressions passed, preserving final PNGs, paths, API events, and exit statuses.
CI-with-TTY checks preserved explicit explore and quiet behavior. All 29 recorded frames passed visual review at the documented widths.
The affected tooling checks pass: 51 picker cases, 50 VS Code protocol cases, and five list/get recording scenes.
CI tests and artifact builds pass at 61e8db6.
Native command/help CI passes on macOS, Linux, and Windows.
CodeQL and the trusted budget check pass at the same head.
Full Windows terminal lifecycle and graphics remain unverified.
API receipt composition, the manpage generator correction, and remaining peer checks keep row 9 incomplete.

Demo

The demo compares real CLI binaries against a synthetic loopback API on macOS.
Before uses da762ff; After uses af738605. Runtime remains unchanged at 61e8db6.
It shows piped JSON parsing, event timing, quiet data preservation, and verbose stderr separation.

The capture validator checked exact bytes, request counts, event timing, and all process statuses.
The terminal replay uses Menlo on macOS. Separate quality captures cover 40 and 100 columns.
These recordings do not establish native terminal-app graphics or Windows lifecycle coverage.

Deterministic output comparison

Before:

Before: forced color and buffered events

After:

After: plain pipes and immediate events

Recording recipe.
Output contract.
Recordings remain outside Git.

@github-actions

github-actions Bot commented Oct 8, 2026 •

Copy link
Copy Markdown

Castiron custom code

Evaluated main: 6fcc478086507571286727084a7d7b162a37ab51.

✅ No new custom-code files detected.

2 mixed files remain; 0 existing customizations changed.

Compared 6fcc47808650 → 61e8db6b7fe8. Generated baselines verified.

2 existing customizations unchanged
  • pkg/cmd/adminorganizationcertificate.go
  • pkg/cmd/audiovoice.go

A changed generated baseline means this report cannot reliably identify which handwritten lines changed.

Inspect the custom-code diff

Download the exact patch produced by this run (requires repository access):

gh run download 37837428714 --repo openai/openai-cli \
  --name castiron-custom-code-37837428714-1 --dir /tmp/castiron-custom-code-37837428714-1
git apply --stat /tmp/castiron-custom-code-37837428714-1/custom-code.patch
cat /tmp/castiron-custom-code-37837428714-1/custom-code.patch

Or reproduce it from an SDK checkout containing the vendored reporter:

git fetch --no-tags origin 6fcc478086507571286727084a7d7b162a37ab51 61e8db6b7fe85b5c1793396f6be6390de99c1d42
python3 scripts/castiron/custom_code_report.py report \
  --base 6fcc478086507571286727084a7d7b162a37ab51 \
  --head 61e8db6b7fe85b5c1793396f6be6390de99c1d42 --fetch --require-head-hash --public \
  --out /tmp/castiron-custom-code-61e8db6b7fe8
cat /tmp/castiron-custom-code-61e8db6b7fe8/custom-code.patch

This is the current full custom patch for mixed files, not an attribution of only the handwritten lines changed by this PR.

Full report and patch

@saioai
saioai changed the base branch from main to codex/flag-descriptions-20261007 October 8, 2026 10:02
@saioai

saioai commented Oct 8, 2026

Copy link
Copy Markdown
Contributor Author

@codex review

Please review commit d5ca5b4ee037221b07cd204c22abe95000405417 against this PR's current base. Check public-command behavior, preserved input/output contracts, errors, and cancellation. Distinguish code defects from the documented validation limits.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 8, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-08T20:00:10.967396Z 61e8db6 New commits
🔒 Security Review ✅ Completed 2026-10-08T19:53:01.460978Z 61e8db6 New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Nice work!

Reviewed commit: d5ca5b4ee0

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@chatgpt-codex-connector

Copy link
Copy Markdown

🛡️ Codex Security Review · Automatically triggered

Security review completed. No security issues were found in this pull request.

Reviewed commit: d5ca5b4ee0

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

@saioai
saioai marked this pull request as ready for review October 8, 2026 17:20
@saioai
saioai requested a review from a team as a code owner October 8, 2026 17:20

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: cd1449a5cb

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".


func configureOutputPolicy(root *cli.Command) {
root.Flags = append(root.Flags,
&cli.BoolFlag{Name: "quiet", Usage: "Suppress optional feedback. Keep data, help, errors and exit status. Overrides verbose.", HideDefault: true},

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Apply quiet policy to generated receipts

When --quiet is used with commands that save output, optional receipts are still printed because the wrapper only adds policy to the context and the generated actions never consult it. For example, openai --quiet files content FILE --output saved.bin still executes fmt.Println(message) in pkg/cmd/file.go:366-369, and openai --quiet @manpages still prints from pkg/cmd/cmd.go:1059; this contradicts the new flag's promise and contaminates stdout. Integrate the policy into the authoritative generator/runtime bridge for these generated actions rather than exposing --quiet before all receipt paths honor it.

AGENTS.md reference: AGENTS.md:L8-L13

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This remains open. The generated binary-save and manpage actions still bypass the handwritten output policy. The authoritative generator correction is prepared, but its source publication and normal generation checkpoints remain blocked. I will preserve generated ownership and keep this requirement marked incomplete until the generated callers and public regressions are verified.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correction to my earlier gate summary: API binary receipts now have a published handwritten fix in #395 at f1e868e. Its request-scoped policy handles completed receipts and returns an empty message to the existing generated print guards. Direct helper callers retain their returned-message contract. No generated edits are needed for those seven API paths. This standalone PR still needs validated composition with that fix. @manpages remains a separate generator correction blocked on source publication. This thread stays open until both paths are integrated and verified.

Comment thread pkg/custom/image_loading_feedback.go
Comment thread pkg/custom/output_policy.go Outdated
Base automatically changed from codex/flag-descriptions-20261007 to main October 8, 2026 17:32

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: af738605da

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread pkg/custom/readable_errors.go
Comment thread pkg/custom/resource_summaries.go

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant