gh-aw upstream follow-up items from PRs (meta-issue)

[davidslater]

Meta-issue for follow-up work on PRs in this repo.

[davidslater]

Additional upstream github/gh-aw work from #686 / PR #744:

• Update buildPrepareDetectionFilesStep in pkg/workflow/threat_detection_steps.go to stage aw-prompts/prompt-template.txt and aw-prompts/prompt-import-tree.json alongside prompt.txt in /tmp/gh-aw/threat-detection/aw-prompts/.
• Add upstream coverage proving all three prompt artifacts are copied into the external detector input directory.
• Once that lands, exercise the integrated features: gh-aw-detection: true path and assert the detector receives a populated ## Prompt Analysis (Trusted vs Untrusted Content) section rather than the degraded fallback.

PR #744 makes missing, empty, or unreadable analysis artifacts visible via ::warning::ERR_VALIDATION and a warning-level prompt_analysis_degraded runlog event, but the production analysis remains degraded until the staging fix lands in github/gh-aw.

[davidslater]

Additional upstream item: stage comment-memory/ into the detection directory

From the comment-memory parity work in #689 / PR #743 (detector-side landed), there is a related upstream (github/gh-aw) follow-up:

Stage comment-memory/ for the external detector.
PR #743 makes threat-detect discover <artifacts-dir>/comment-memory/*.md, add a {COMMENT_MEMORY_FILES} prompt section, and warn (ERR_VALIDATION) on inspection failure. But the detector only sees files that gh-aw actually places in the staged detection directory. While auditing #689 I could not find a step that copies comment-memory/ into /tmp/gh-aw/threat-detection/ — buildPrepareDetectionFilesStep doesn't copy it — so the directory may be absent at detection time (the inline gh-aw path likely also reports "No comment-memory files found" today).

Upstream work needed:

1. Verify against a real gh-aw run whether comment-memory/ is present in the staged detection directory.
2. If absent, copy comment-memory/ in buildPrepareDetectionFilesStep (mirroring how aw-prompts/, agent_output.json, and patch/bundle files are staged) so the detector can analyze it. Note replay-detection.yml in this repo already stages comment-memory/ for replays, so that path is covered.

Verification / follow-up in this repo

3. Smoke assertion that comment-memory content reaches the detector — e.g. seed a comment-memory/*.md file with injection/secret-like content and assert the detector flags it (only meaningful once item 2 lands upstream).

References

• This-repo change: PR feat(artifacts): analyze comment-memory files for threats #743, issue Parity: comment-memory files are not analyzed (missing {COMMENT_MEMORY_FILES} prompt section and artifact discovery) #689
• Detector discovery: pkg/artifacts/artifacts.go (loadCommentMemory), prompt placeholder {COMMENT_MEMORY_FILES} in pkg/detector/prompts/threat_detection.md
• Spec: specs/threat-detection-spec.md → TD-18a; specs/usage-spec.md → U-11

[davidslater]

Upstream work needed for #692 (conclude diagnostics parity)

Adding a second cluster of upstream items, from the work on #692 (threat-detect conclude now emits full diagnostic output at parity with parse_threat_detection_results.cjs).

The problem: actions/setup/sh/conclude_threat_detection.sh short-circuits the two most diagnostically valuable cases in bash and exits before threat-detect conclude is ever invoked:

if [ "${RUN_DETECTION:-false}" != "true" ]; then
  ... echo conclusion=skipped ...; exit 0
fi

if [ ! -f "${RESULT_FILE}" ]; then
  ... echo "::warning::${result_message}..."; exit 0   # or exit 1
fi

threat-detect conclude --result-file "${RESULT_FILE}"   # only reached on the happy path

So on the features: gh-aw-detection: true path, the new missing-result diagnostics — recursive directory listing, detection-log line/byte stats, and the echo of every THREAT_DETECTION_STATUS:/THREAT_DETECTION_RESULT: marker line — are dead code, which is exactly the failure mode where they matter most. The wrapper's replacement is strictly weaker: it greps only the last THREAT_DETECTION_STATUS: line, with no directory listing and no per-field verdict breakdown.

Upstream items (in github/gh-aw)

5. Stop short-circuiting the missing-result-file case in conclude_threat_detection.sh.
   Delete the if [ ! -f "${RESULT_FILE}" ] block and let threat-detect conclude handle it. The detector already implements the identical contract — agent_failure reason, warn-vs-strict via GH_AW_DETECTION_CONTINUE_ON_ERROR, ERR_SYSTEM: ❌ Detection result file not found at: <path> message, and the mustFail hard-fail when DETECTION_AGENTIC_EXECUTION_OUTCOME=failure — plus the full diagnostics. This also removes a duplicated (and drifting) reimplementation of the conclusion contract in bash.

6. Stop short-circuiting the RUN_DETECTION != "true" skip case.
   Same reasoning: conclude already emits conclusion=skipped / success=true / reason=. Two notes on the current bash version:

   • It writes only GITHUB_OUTPUT and never exports GH_AW_DETECTION_CONCLUSION / GH_AW_DETECTION_REASON to GITHUB_ENV, so the intercepted paths silently violate U-18. Delegating to conclude fixes this for free.
   • Its continue_on_error default (:-true) and lowercase normalization differ subtly from the detector's != "false" check; collapsing to one implementation removes the divergence risk.

7. Forward DETECTION_LOG_FILE via the new --detection-log flag.
   The wrapper already supports a DETECTION_LOG_FILE env override. Once items 5–6 land, pass it through:

   threat-detect conclude --result-file "${RESULT_FILE}" --detection-log "${DETECTION_LOG_FILE}"

   ✅ No change needed for the default case — I verified the detector's default (detection.log beside the result file) is byte-identical to both constants.ThreatDetectionLogPath (/tmp/gh-aw/threat-detection/detection.log) and the wrapper's ${RESULT_DIR}/detection.log. The flag only matters when a caller overrides the log location.

8. Optional: pass --log-file to capture the conclusion in JSONL.
   conclude now mirrors conclude_start, conclude_verdict, conclude_directory_listing, conclude_detection_log, and conclude_outcome into the JSONL run log when --log-file is set. If gh-aw already uploads a detection run-log artifact, wiring this in makes the conclusion machine-queryable alongside the detection run itself.

Not blocked

Items 5–8 are pure upgrades: nothing in #692 regresses the current behavior, since the detector's happy path (the only path the wrapper actually reaches today) gained the banner, the echoed inputs, and the per-field verdict breakdown, and those do show up in the job log now. The items above are about unlocking the failure-path diagnostics, which is where the debugging pain in #692 was reported.

Verification follow-up in this repo

9. Once items 5–6 land, extend a smoke workflow to force a missing/malformed detection_result.json and assert the directory listing + detection-log stats appear in the conclude step's job log.

_{Refs: this-repo change #692; spec TD-20c (specs/threat-detection-spec.md) and U-18a (specs/usage-spec.md).}

[davidslater]

Cross-linking related upstream work: item 3 above ("Stop discarding the rendered prompt / related step-summary issue") is the same root problem tracked in #696, which I addressed on this repo's side in PR #739.

What's now done in this repo (PR #739):

• threat-detect supports --step-summary <path> (defaults to $GITHUB_STEP_SUMMARY) and appends a collapsible block containing the prompt it actually rendered (template + PROMPT_ANALYSIS + resolved engine/model/retries).
• threat-detect conclude supports its own --step-summary and appends a collapsible verdict block (per-category booleans, reasons, conclusion, reason code) for every terminal outcome.
• Both blocks HTML-escape untrusted content and are best-effort (a write failure never changes the exit code).

Upstream work still needed in github/gh-aw (unchanged from #696 item 3 / this issue's item 3):
On the external-detector path (features: gh-aw-detection: true), buildThreatDetectionAnalysisStep (pkg/workflow/threat_detection_steps.go) still runs the Setup threat detection github-script step and writes its own rendered prompt to the step summary via core.summary.addRaw(...). Now that threat-detect writes the prompt it actually used, gh-aw should either:

• skip that github-script step's summary write on the external-detector path, or
• stop rendering/writing that prompt block there entirely,

so the step summary never shows two prompts (one unused, one real) for the same detection run. This is purely a gh-aw repo change — no further action needed in gh-aw-threat-detection for it.

No other upstream work is needed from PR #739 beyond this.

[davidslater]

Additional upstream work from #697 / PR #746:

• In github/gh-aw, update buildUploadDetectionArtifactStep in pkg/workflow/threat_detection_external.go to upload /tmp/gh-aw/threat-detection/detection-runlog.jsonl alongside detection_result.json and detection.log. PR feat(runlog): persist structured detection logs #746 now defaults the runlog to that path whenever --output is set, and the existing AWF ThreatDetectionDir read-write mount already preserves it on the host.
• Add/update compiler coverage (notably threat_detection_isolation_test.go) and define a shared path constant if appropriate.
• Verify gh aw audit recognizes/surfaces the JSONL after artifact download; add ingestion/reporting support if it currently only inventories the file.

No existing github/gh-aw issue or PR referencing detection-runlog.jsonl was found, so this remains untracked upstream work.

[davidslater]

Additional upstream follow-up from #702 / PR #741

No functional installer or compiler change is required in github/gh-aw: actions/setup/sh/install_threat_detect_binary.sh already maps Linux and Darwin on amd64/arm64 to the four asset names now published by PR #741.

Two small upstream maintenance items remain:

1. Update the installer's # Platform support header, which still says Linux-only, to document macOS x64/arm64 support.
2. Add installer tests covering the Linux and Darwin architecture-to-asset mappings (there is currently no install_threat_detect_binary test). This repo's new scheduled parity check validates the mappings against release-targets.txt, but an upstream test would also protect installer behavior locally.

The macOS assets remain unsigned/not notarized; the installer checksum-verifies them before execution, as documented in PR #741.

[davidslater]

Adding a cross-repo upstream item identified while working issue #694 / PR #738 (conclude-logic parity):

Upstream work needed (in github/gh-aw)

Fix or simplify actions/setup/sh/conclude_threat_detection.sh. I fetched the current version of this script from github/gh-aw and confirmed it still has the bugs described in #694:

• The RUN_DETECTION != true and missing-result-file branches are handled inline in the shell script and never delegate to threat-detect conclude, even though conclude.go on this side already handles those cases correctly (skip, missing file → agent_failure, unreadable file → agent_failure, malformed file → parse_error), including writing reason/conclusion outputs and GH_AW_DETECTION_* env vars consistently in both warn and strict mode.
• Concretely, the strict-mode (continue_on_error != true) missing-file branch does:

  echo "ERR_SYSTEM: ❌ ${result_message}"
  exit 1

  without ever writing conclusion=failure / reason=agent_failure to $GITHUB_OUTPUT, and without exporting GH_AW_DETECTION_CONCLUSION/GH_AW_DETECTION_REASON. Downstream consumers of needs.detection.outputs.detection_reason see an empty string in this case, which (per Parity: conclusion logic is duplicated in conclude_threat_detection.sh and threat-detect conclude, with divergent (and partly broken) semantics #694) can render a strict-mode infrastructure failure as a generic/confusing threat message instead of a clear agent-failure message.

Suggested fix: have the shell script always delegate to threat-detect conclude --result-file "${RESULT_FILE}" (dropping the shell's own missing-file special-casing entirely), since the Go implementation already handles missing/unreadable/malformed files correctly and consistently across warn/strict mode. This would eliminate the duplicated-and-diverging logic at the root, per proposed work item (1) in #694.

This is separate from the workflow-context plumbing (WORKFLOW_NAME/WORKFLOW_DESCRIPTION/CUSTOM_PROMPT) items 1–3 already tracked above — flagging it here since this issue is the active tracker for github/gh-aw-side follow-ups. Filing as a note rather than a new issue in case you'd prefer to fold it in or split it out.

[davidslater]

Created by GitHub Ace · View Session

Tracks the upstream (github/gh-aw) and cross-repo work needed to complement the this-repo changes in #688 / PR #742.

PR #742 added a plumbing-independent way for the detector to receive workflow context (--workflow-name, --workflow-description, --custom-prompt, --custom-prompt-file) plus diagnostics (prompt_built run-log fields and a stderr Prompt context: line). But on the features: gh-aw-detection: true path, the three env vars still never reach threat-detect, so safe-outputs.threat-detection.prompt remains silently ignored until gh-aw is updated.

Upstream work needed (in github/gh-aw)

1. Pass the workflow context to the external detector execution step.
   buildExternalDetectorExecutionStep (pkg/workflow/threat_detection_external.go) builds its env exclusively from engine.GetExecutionSteps(...). Add WORKFLOW_NAME, WORKFLOW_DESCRIPTION, and (conditionally) CUSTOM_PROMPT to that step's env block — mirroring what buildThreatDetectionAnalysisStep (pkg/workflow/threat_detection_steps.go) already sets on the Setup threat detection github-script step via c.buildWorkflowContextEnvVars(data) + CUSTOM_PROMPT: %q.

2. Ensure the vars survive the AWF env filter.
   Add WORKFLOW_NAME, WORKFLOW_DESCRIPTION, and CUSTOM_PROMPT to the AWF env passthrough list (ComputeAWFExcludeEnvVarNames) so they are not stripped when detection runs under AWF.

   • Alternative / more robust: instead of relying on env passthrough, have gh-aw pass the custom prompt to the detector via the new --custom-prompt-file flag (writing the prompt to a file), so it cannot be dropped by env filtering at all. This is the preferred approach now that PR feat(cli): add explicit workflow-context flags and prompt diagnostics #742 provides the flag.

3. Stop discarding the rendered prompt (related step-summary issue).
   The setup step renders a prompt with these values and writes it to the step summary, but threat-detect never reads GH_AW_PROMPT, so that rendered prompt is discarded. Decide whether gh-aw should stop rendering it there, or whether the detector should consume it.

Verification / follow-up in this repo (github/gh-aw-threat-detection)

4. Smoke workflow asserting the prompt reaches the detector.
   Add (or extend) a smoke workflow with safe-outputs.threat-detection.prompt set that asserts the custom text is applied — e.g. by checking the prompt_built run-log custom_prompt_source / custom_prompt_bytes fields or the stderr Prompt context: line added in PR feat(cli): add explicit workflow-context flags and prompt diagnostics #742. This is only meaningful once items 1–2 land upstream.

Context / references

• This-repo change: PR feat(cli): add explicit workflow-context flags and prompt diagnostics #742, issue Parity: CUSTOM_PROMPT / WORKFLOW_NAME / WORKFLOW_DESCRIPTION never reach threat-detect, so threat-detection.prompt is silently ignored #688
• Spec: specs/threat-detection-spec.md → TD-22-flags
• Diagnostics to assert against: prompt_built run-log event fields (workflow_name, workflow_name_defaulted, custom_prompt_applied, custom_prompt_source, custom_prompt_bytes) and the stderr Prompt context: line.

[davidslater]

Upstream work needed for PR #737 (issue #693)

PR #737 taught threat-detect conclude to consult a new --detection-log (default <result-file-dir>/detection.log) and map the terminal THREAT_DETECTION_STATUS: reason=<reason> line onto the gh-aw job-output reason contract when the result file is missing:

status reason	gh-aw reason
invalid_report_exhausted	parse_error
output_write_error	parse_error
engine_error	agent_failure
cancelled	agent_failure
config_error	agent_failure
absent / unrecognized	agent_failure

This mapping is currently dead code in production. In github/gh-aw, actions/setup/sh/conclude_threat_detection.sh short-circuits the missing-result-file case itself — it never calls threat-detect conclude at all when detection_result.json is absent:

if [ ! -f "${RESULT_FILE}" ]; then
  if [ "${continue_on_error}" = "true" ]; then
    echo "reason=agent_failure" >> "${GITHUB_OUTPUT}"   # hardcoded, no log consulted
    exit 0
  fi
  exit 1   # strict mode: no reason/conclusion written at all
fi
threat-detect conclude --result-file "${RESULT_FILE}"   # only reached when the file exists

So for PR #737's mapping to actually take effect, github/gh-aw needs to:

1. Stop short-circuiting the missing-file case in conclude_threat_detection.sh and always delegate to threat-detect conclude --result-file <path> --detection-log <path> (the flag defaults to <result-file-dir>/detection.log if omitted, but gh-aw should pass it explicitly to be robust to path layout changes).
2. Ensure detection.log (the detection run's captured stderr/log, distinct from the JSONL --log-file trace) is actually written to a location conclude can read — i.e. the execution step's stderr must be captured to a file at that path before the conclude step runs.
3. Fix the strict-mode bug where the shell's exit 1 branch currently writes no reason/conclusion output at all (a pre-existing bug, not introduced by fix(conclude): map THREAT_DETECTION_STATUS reason for missing result files #737, but it means strict-mode infra failures render as [!CAUTION] agentic threat detected instead of [!WARNING] threat detection engine error).

This is the same root cause already tracked in this repo as #694 ("conclusion logic is duplicated in conclude_threat_detection.sh and threat-detect conclude, with divergent — and partly broken — semantics"), which should be resolved as part of collapsing to a single implementation. Until #694's item 1 lands upstream, #737's richer reason mapping only benefits direct/replay callers of threat-detect conclude, not gh-aw-compiled workflows using features: gh-aw-detection: true.

[davidslater]

Upstream work needed for #698 (AI-credit reporting / max-ai-credits parity)

From the parity review in #698. The investigation (see the findings comment on #698) concluded that both halves of the issue are already satisfied on the shipping external path by the AWF API proxy: apiProxy.maxAiCredits enforces the cumulative credit cap (covering --retries), and parse_token_usage.cjs sources aic from the proxy's …/api-proxy-logs/token-usage.jsonl — not from threat-detect's detection.log/stdout, so engine-flag drift cannot break it. The detector-side docs were corrected in PR #748. The remaining items are upstream / verification only.

Upstream items (in github/gh-aw)

10. Add a smoke assertion for a non-empty aic on the standalone detection path. This is Parity: AI-credit reporting (aic) unverified and max-ai-credits budget not enforced on the external detector path #698 item 5. aic is the detection job output (needs.detection.outputs.aic → GH_AW_THREAT_DETECTION_AIC), which the agentic smoke step in this repo cannot observe. The assertion has to live in the compiled workflow that gh-aw generates (e.g. a step in a job that needs: detection asserting the output is present/non-empty, or a compiler-level test over the generated lock). Belongs upstream, not in smoke-*-standalone.md here.

11. Decide whether the non-AWF / non-token-steering detector path needs credit accounting at all. Both maxAiCredits enforcement and aic derivation depend on apiProxy.enabled + enableTokenSteering: true. A plain ./bin/threat-detect invocation (no proxy) has no credit cap and produces no token-usage.jsonl. If gh-aw ever runs the external detector without the API proxy, it would need either to keep it out of scope explicitly or to have threat-detect emit its own agent_usage.json from the engine's usage records (issue item 2's fallback). Today the shipping path is always AWF+proxy, so this is a "confirm and document as out-of-scope" item rather than a code change.

Not blocked

Nothing in #698 regresses current behavior — the credit cap and aic both work on the AWF path today. Items 10–11 are verification/coverage and a scoping decision, not fixes.

Verification follow-up in this repo

12. Once item 10 lands upstream, the corresponding smoke assertion is exercised in CI; no additional this-repo change is expected unless item 11 concludes threat-detect should emit agent_usage.json itself.

_{Refs: this-repo docs change PR #748; issue #698. Source of truth: actions/setup/js/parse_token_usage.cjs (proxy token-usage.jsonl → aic, writes /tmp/gh-aw/agent_usage.json); apiProxy.maxAiCredits in the compiled smoke-*-standalone.lock.yml.}

[davidslater]

Created by GitHub Ace · View Session

Summary

Under AWF, the standalone detection job cannot resolve Copilot model aliases such as auto, so detection fails with:

[copilot-harness] attempt 1: refreshed awf-reflect does not include model 'auto' — treating as non-retryable
400 The requested model is not supported.
THREAT_DETECTION_STATUS: reason=engine_error exit=2

This is an upstream github/gh-aw compiler issue, not a threat-detect binary issue. It should be fixed in the gh-aw workflow compiler. This repo has a local workaround (see below), but the proper fix belongs upstream so it works for all consumers using their engine's default detection model.

Root cause

1. The detection engine runs inside AWF. With gh-aw 1.0.75, Copilot is launched through copilot_harness.cjs.
2. The harness resolves/validates the requested model against the api-proxy's awf-reflect registry (GET http://api-proxy:10000/reflect), not by reading awf-config.json directly.
3. The awf-reflect model_aliases map is populated from apiProxy.models in the AWF config that launched the sandbox.
4. The gh-aw compiler emits the agent job's AWF config with the full apiProxy.models map, but the detection job's AWF config omits apiProxy.models entirely.
5. Result: detection's awf-reflect has no auto alias, COPILOT_MODEL=auto is passed verbatim to the Copilot CLI, and the CLI returns 400 The requested model is not supported.

The detection job default is COPILOT_MODEL: ${{ vars.GH_AW_MODEL_DETECTION_COPILOT || vars.GH_AW_DEFAULT_MODEL_COPILOT || 'auto' }} (with older compiler versions; newer versions default to claude-sonnet-5). Any alias — not just auto — that requires model_aliases is unresolvable in the detection job.

Evidence

• Agent awf-reflect.json: model_aliases.auto = ["copilot/auto", "large"] → resolves (copilot model alias resolution: 'auto' -> 'claude-sonnet-5').
• Detection detection.log: refreshed awf-reflect does not include model 'auto'.
• Comparing the two AWF configs in the compiled lock (smoke-copilot-standalone-latest.lock.yml):
  • Agent config (apiProxy): includes the full "models": { ... "auto": ["copilot/auto","large"], ... } map.
  • Detection config (apiProxy): {"enabled":true,"enableTokenSteering":true,"maxRuns":500,"maxAiCredits":...,"maxCacheMisses":5} — no models key.
• The last passing baseline (older gh-aw, threat-detect v0.3.1) invoked copilot directly, without the harness, so auto was never validated. The failure is a regression exposed once the detection path started using copilot_harness.cjs with strict awf-reflect model validation.

Why PR #684 (in this repo) did not fix it

gh-aw-threat-detection#684 made threat-detect export GH_AW_AWF_CONFIG_PATH pointing at the mounted $RUNNER_TEMP/gh-aw/awf-config.json so the harness could read the alias map. That does not help because:

1. The harness resolves models via the running api-proxy's awf-reflect, not by reading the config file at spawn time.
2. The mounted detection config also lacks apiProxy.models, so there is no alias map to read anywhere in the detection job.

The api-proxy's reflect registry is fixed at container startup from the config that launched AWF, so the only durable fix is to put apiProxy.models into the detection AWF config.

Proposed upstream fix (github/gh-aw)

When the compiler builds the detection job's AWF config, include the same apiProxy.models alias map it already emits for the agent job (see buildThreatDetectionWorkflowData / the external-detector config generation in pkg/workflow/threat_detection_external.go and the AWF config/api-proxy model-map generation). Then detection's awf-reflect will expose the aliases and the harness can resolve auto (and any other alias) exactly as the agent job does.

Acceptance criteria:

• Detection job's AWF config contains apiProxy.models (parity with the agent job).
• With COPILOT_MODEL=auto (or any alias), the detection copilot harness resolves the alias via awf-reflect and does not emit 400 The requested model is not supported.
• Add/extend a compiler test asserting the detection AWF config includes apiProxy.models.

Local workaround already applied in this repo

Pinned a concrete detection model via safe-outputs.threat-detection.model: claude-sonnet-5 in the copilot standalone smoke workflows (smoke-copilot-standalone.md and smoke-copilot-standalone-latest.md, plus the patched .lock.yml detection COPILOT_MODEL fallback), so detection no longer depends on alias resolution. This unblocks the smokes but does not fix the underlying gap for consumers relying on the engine's default (auto) detection model — hence this upstream issue.

References

• Failing run: https://github.com/github/gh-aw-threat-detection/actions/runs/30582185023/job/91005457742
• Superseded fix attempt: gh-aw-threat-detection#684

[davidslater]

Upstream gh-aw work needed for threat-detection artifact parity (from #747)

PR #747 makes threat-detect consume aw_info.json as bounded, untrusted activation context and inventory all artifacts. During that work I found the detection-job staging step in gh-aw does not stage the files the detector now reads, so parts of the feature are inert end-to-end until upstream changes land.

1. REQUIRED — stage aw_info.json into the detection dir

pkg/workflow/threat_detection_steps.go → buildPrepareDetectionFilesStep copies only:

• aw-prompts/prompt.txt
• agent_output.json
• aw-*.patch
• aw-*.bundle

into /tmp/gh-aw/threat-detection/. It never copies aw_info.json, so the new {ACTIVATION_CONTEXT} prompt section is empty in the integrated flow. Fix: add a copy of /tmp/gh-aw/aw_info.json (guarded for existence) into the staged detection dir, then recompile affected .lock.ymls.

2. NOTE — pre-existing gap: prompt-template.txt / prompt-import-tree.json

The same step also omits aw-prompts/prompt-template.txt and aw-prompts/prompt-import-tree.json, which the detector already documents/consumes for the trusted-template vs untrusted-input split. That split is likewise inert integrated until these are staged. Optional: also stage experiments/ and comment-memory/ if they should be within the detection surface (currently inventory-only in the detector).

3. CAUTION — GITHUB_STEP_SUMMARY writability under AWF

#747 writes an artifact-inventory table to GITHUB_STEP_SUMMARY and (per spec TD-20c) treats a failed write as a config error (exit 2). In the AWF sandbox, if GITHUB_STEP_SUMMARY is set but its path isn't a writable mount, detection would fail closed. Upstream should ensure the detection job runs with GITHUB_STEP_SUMMARY on a writable mount (or we soften the detector to warn instead of erroring).

4. Reminder — smokes pin to a promoted release

The *-standalone smokes download a promoted threat-detect release, so none of this is exercised end-to-end until a release is cut and promoted. Sequencing: land #747 → cut/promote a release → land the gh-aw staging change → recompile locks.

[davidslater]

Upstream (gh-aw): pipe detection env vars into the external detector execution step

From #751 (parity with gh-aw#49415).

gh-aw#49415 taught the detection setup step to honor GH_AW_DETECTION_CONTINUE_ON_ERROR, and #751 mirrors that validation inside threat-detect so the Go detector no longer depends on the (being-retired) JS setup step. But gh-aw only pipes those env vars into the JS setup and conclusion steps — not into the step that actually runs the binary.

Verified against a freshly compiled .github/workflows/smoke-copilot-standalone.lock.yml (gh-aw v0.84.3, i.e. post-#49415):

Step	GH_AW_DETECTION_CONTINUE_ON_ERROR	HAS_PATCH	WORKFLOW_NAME
Setup threat detection (JS)	✅ L1357	✅ L1356	✅ L1354
Execute threat detection with AWF (runs threat-detect)	❌	❌	❌
Conclude threat detection	✅ L1467	—	—

The invocation also passes no equivalent flags: set -- --engine copilot --output /tmp/gh-aw/threat-detection/detection_result.json ... /tmp/gh-aw/threat-detection.

Asks

In gh-aw's buildExternalDetectorExecutionStep (pkg/workflow/threat_detection_external.go), add to the execution step env:

GH_AW_DETECTION_CONTINUE_ON_ERROR: <same expr already emitted for setup/conclude>
HAS_PATCH: ${{ needs.agent.outputs.has_patch }}

Related gap in the same place — the prompt-context vars are also setup-only, so every external detection run renders its prompt with "Unnamed Workflow" and logs workflow_name_defaulted=true:

WORKFLOW_NAME: <workflow name>
WORKFLOW_DESCRIPTION: <workflow description>
CUSTOM_PROMPT: <threat-detection.prompt, when configured>

(Alternatively gh-aw can pass --workflow-name / --workflow-description / --custom-prompt-file, which the detector already supports and which survive env-passthrough filters.)

No AWF change is needed — the detection step runs awf --env-all, so step-level env reaches the sandboxed process.

Impact until then

No regression: with the vars absent the detector sees warn mode (the documented default) and an unset HAS_PATCH, i.e. its previous lenient behavior. The gap is that a workflow configured with threat-detection: continue-on-error: false gets strict artifact validation in gh-aw's JS setup but not in the detector, so the two stages disagree — which starts to matter as the JS setup is retired. Our own smoke workflows set continue-on-error: false, so they'll exercise the strict path automatically once the env var is piped through.

Optional local follow-up here

.github/workflows/detection-only.yml is hand-maintained and hardcodes GH_AW_DETECTION_CONTINUE_ON_ERROR: "true" on the conclude step only; we could add both vars to its detection execution step to exercise the new path without waiting on upstream.