Skip to content

GitHub identity Phase 0: stop the token leaks (#184) - #186

Draft
fedacking wants to merge 12 commits into
mainfrom
github-identity-phase0
Draft

fedacking wants to merge 12 commits into
mainfrom
github-identity-phase0

Conversation

@fedacking

@fedacking fedacking commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor

Closes #184. This is Phase 0 of #181, "stop the leaks", done without the GitHub App.

Exit criteria

  • No user token in any container.
  • CSRF-safe login.
  • Sessions stop working within minutes of a user losing org access or their role.

There is one commit per item, so it can be reviewed commit by commit. 0.2 has two: tokens out of previews, then isolating the preview build itself. 0.5 has two: token encryption, then moving the dashboard secrets out of persistent storage.

Commit Item
0.1 No tokens in agent containers. New host-side repo_sync.rs fetches into a root-owned bare cache, with the token passed through GIT_CONFIG_* env (never in argv or on disk), and ships git bundles into agents. Agent remotes are plain URLs, and existing containers are scrubbed at startup. Agents no longer push: delivery is always createCommitOnBranch, and auto_create_pr=false now only suppresses the PR.
0.2 No user token in preview containers. preview.sh fetches on the host and streams a bundle into the container, where the preview user checks it out. /etc/preview-token and credentialed .git/config are gone. preview scrub-tokens cleans existing previews and runs on deploy and before each update. hostSecrets lookup is now exact-match and refuses infra keys (GITHUB_*, WEBHOOK_*, PREVIEW_*, ADMIN_SSH_KEY, ALLOWED_REPOS).
0.2 Untrusted preview-config.nix can't reach host secrets. The repo's config used to be evaluated as root with --impure, so it could readFile /run/dashboard-secrets, /var/secrets or any host file, and copy the contents into /nix/store (readable by everyone and mounted in every container) or send them out. Its fixed-output builds could also reach loopback, where /internal/secrets (any repo's decrypted secrets), the webhook's /internal/token and Caddy's admin API trust the caller. Now evaluation runs through systemd-run as a throwaway DynamicUser: Nix's restrict-eval is on, there's no network (IPAddressDeny=any), the environment is empty, and the secret paths are hidden. Restricted mode alone isn't enough, because getFlake "path:/dir" still copies any readable host dir into the store. nixpkgs comes from /etc/nixos/flake.lock as a locked store path, so closures are byte-identical to before. A tekton-nixbld firewall chain rejects traffic from nixbld build users to loopback and to private and link-local addresses, DNS excepted. preview.sh refuses to build without the chain. Swap is pinned off, so the tmpfs secrets can't be paged out.
0.3 OAuth state on login. A 10-minute oauth_state cookie, checked with a constant-time compare. Cancelling on GitHub now redirects home.
0.4 Sessions follow GitHub reality. session_version in the JWT and DB, and role always read from the DB. Org membership is re-checked after 10 minutes, or 2 minutes for member and admin actions. Losing membership or having the token revoked ends every session. Transient GitHub errors get a bounded grace (1 h, or 10 min for privileged actions) and never revoke. Role changes bump the version.
0.5 Encrypt stored GitHub tokens. Legacy plaintext tokens are encrypted and NULLed at startup. Every read goes through auth::load_github_token.
0.5 Secrets live in 1Password, not on disk. SECRETS_ENCRYPTION_KEY, JWT_SECRET and GITHUB_CLIENT_SECRET live in 1Password and on the server only in /run/dashboard-secrets/secrets.env (tmpfs). The unit requires that file, so the dashboard won't start without it. There is no 1Password CLI: an authorized op session would let anything in that terminal read every vault. Instead, server-config/dashboard-secrets.sh set <server> asks for each value at a hidden prompt on the server, and you paste it from the 1Password app. It writes the file atomically and refuses an empty or partial one. It also refuses a SECRETS_ENCRYPTION_KEY that doesn't match the running dashboard's key, unless you pass --new-key. Any pasteable value works: they're written double-quoted with systemd escaping. Values never reach argv or shell history, never touch the laptop's disk, and aren't echoed. check-live, copy-live and verify-key support the move from the old setup without printing values. At startup the backend refuses a key that decrypts none of the oldest stored values, before encrypting legacy tokens, and removes the three from its environment so child processes (agent, preview, git, chromium) don't inherit them. deploy.sh checks before restarting the dashboard: the file must be present and complete, the unit must load it, and no copies may remain in dashboard.env or old key files. After the restart, it checks the service actually came up. server-setup.sh writes the /run file instead of dashboard.env, and refuses to mint new secrets on an existing install. Threat model: this protects DB dumps, backups, disk images and file reads. It doesn't protect live root or memory reads, which nothing local can.
0.6 Guard intake and Sentry. Before each poll or spawn, run_as_user must still be an org member with a usable token. Otherwise the source is auto-disabled, with a visible reason and a system audit event. Without a label filter, only OWNER/MEMBER/COLLABORATOR issues are taken. Issue text goes inside a per-prompt random <untrusted_issue_*> fence, with marker neutralization and single-pass template substitution. Prompts are rebuilt when the task spawns.
0.7 Provenance. Commits get Tekton-Task and Co-authored-by trailers. PR bodies end with a footer: requested by, a task link, and the model. A new tasks.model column records the model.

Breaking changes and deploy notes

  • Preview configs: each target repo's preview-config.nix must drop its git clone/fetch step, because the code is now provided at /home/preview/app (PREVIEW_CODE_PROVIDED=1). Old configs keep working only for public repos.
  • Preview build isolation: manual step on prod. deploy.sh never copies configuration.nix, so copy the networking.firewall.extraCommands/extraStopCommands blocks and the swap settings into /etc/nixos/configuration.nix on the server, then run ./deploy.sh <server> nix. Until then, previews refuse to build, and deploy.sh warns. See "Isolating preview builds" in docs/deployment-guide.md.
  • Preview configs can't fetch at evaluation time: builtins.fetchTarball/fetchGit/fetchurl, readFile of host paths and getEnv now fail. Derivation fetchers (pkgs.fetchurl, fetchFromGitHub) still work.
  • Rotate tokens after deploy: old previews exposed the server GITHUB_TOKEN in /var/secrets/preview.env, and user OAuth tokens, to app code. Scrubbing doesn't undo that exposure.
  • Old backups: backups and WAL taken before the upgrade still contain live plaintext tokens. Run VACUUM FULL users; after the first start.
  • Dashboard secrets: manual upgrade on prod. Follow "Upgrading an existing server" in docs/deployment-guide.md:
    1. Run dashboard-secrets.sh check-live <server> to confirm the live values can be entered at the prompt. It never prints them.
    2. Create the 1Password item key-tekton-00 in the Tekton vault, with fields SECRETS_ENCRYPTION_KEY, JWT_SECRET and GITHUB_CLIENT_SECRET.
    3. Copy the existing key and client secret in, by running dashboard-secrets.sh copy-live <server> NAME | pbcopy and pasting in the app. Don't generate a new key, because existing secrets are encrypted with it.
    4. Generate JWT_SECRET in the app. A new one signs everyone out.
    5. Run dashboard-secrets.sh verify-key <server> and paste the key. Don't continue until it says OK.
    6. Run dashboard-secrets.sh set <server> and paste the three fields.
    7. Hand-edit /etc/nixos/configuration.nix on the server to add the EnvironmentFile, since deploy.sh doesn't copy that file.
    8. Remove the three lines from /var/secrets/dashboard.env.
    9. Run ./deploy.sh <server>.
  • After a reboot, the dashboard stays down until someone runs dashboard-secrets.sh set <server> and pastes the three values again, because tmpfs is wiped. This is accepted for now. If reboots become frequent, we'll revisit (TPM-sealed systemd credentials are the likely next step).
  • Losing the key: if the 1Password item is lost, users must sign in again, every intake and Sentry source gets auto-disabled, and repo secrets, AI keys and Sentry tokens must be re-entered. Rolling back past 0.5 breaks tasks until users sign in again.
  • Local dev: make deps writes a random SECRETS_ENCRYPTION_KEY into new .env files. Existing .env files that only have SECRETS_ENCRYPTION_KEY_FILE need a SECRETS_ENCRYPTION_KEY line, plus JWT_SECRET and GITHUB_CLIENT_SECRET set to non-empty values.
  • Other behaviour changes:
    • Intake tasks with auto_create_pr off now get a pushed branch and previews.
    • Role changes sign the user out.
    • Branch names containing ' or whitespace are rejected by preview.sh.
    • PREVIEW_*-named hostSecrets are no longer forwarded.

Verification

  • Automated:
    • cargo fmt --check, cargo clippy --all-targets -D warnings and cargo test pass (60 tests). They include real-git file:// tests for clone, reopen, thin-bundle fallback and tags, plus a guard that x-access-token and extraHeader appear only in repo_sync.rs.
    • Frontend npm run lint (0 errors) and npm run build pass.
    • bash -n and shellcheck pass on preview.sh and deploy.sh. bash -n passes on server-setup.sh and dashboard-secrets.sh.
    • The startup key check was run against a local Postgres 18:
      • The right key starts.
      • A wrong key refuses, even when newer rows use the wrong key.
      • One bad row among good ones still starts.
      • e2e/seed.sql passes with the CI key.
    • dashboard-secrets.sh and the deploy.sh preflight were run in a local expect/pty harness, with macOS bash 3.2 on the laptop side, bash 5 for the server side, and systemctl stubbed. It covered:
      • Byte-exact round trips of values with spaces, ', ", $, backticks and backslashes.
      • Bracketed and multi-line pastes.
      • Live-key mismatches.
      • Ctrl-C mid-prompt, which leaves no temp file.
      • No pasted value ever showing in the transcript.
      • Old key files, a unit that doesn't load the file, and a crash loop after set.
    • The systemd env-file quoting rules were checked against an independent reimplementation, not real systemd.
    • Preview build isolation was checked with Nix 2.33 on macOS, using the real locked_nixpkgs_ref. The template config gives the same toplevel .drv as the old unrestricted path, and previewMeta evaluates. readFile of host paths, every URL fetcher (loopback included) and getEnv are refused. getFlake "path:/dir" still copies a host dir under restricted mode alone, so on Linux the DynamicUser layer is what stops it.
  • Not yet run:
    • The Playwright e2e suite.
    • dashboard-secrets.sh against a real server (Linux tty handling, real systemd parsing), and the 9-step secrets upgrade.
    • Everything on a staging host: agents and previews on a private repo, nixos-container run stdin, runuser in the container, scrub-tokens, the membership revocation flow, intake auto-disable, and the GitHub App webhook path. Also the systemd-run sandbox (daemon access as a DynamicUser, --pipe output), the tekton-nixbld chain, and a canary config that hashes /run/dashboard-secrets/secrets.env (it must fail), plus a fixed-output build that curls 127.0.0.1:3200 (it must be rejected).
    • These checks are in the plan's manual verification table.
  • To confirm on staging: whether a run_as_user with private org membership shows up as CONTRIBUTOR on Sentry-created issues, which the author filter would drop.

Follow-ups (out of scope)

  • WebSocket routes /ws/tasks/{id} and /ws/logs/{slug} have no auth.
  • generate_pr_body runs claude --dangerously-skip-permissions on the host with untrusted text.
  • nix build --impure on a repo-controlled preview-config.nix runs as root and can read /var/secrets/*.
  • Stored token ciphertext isn't bound to the user (no AAD). The same is true of repo secrets.
  • GITHUB_WEBHOOK_SECRET is still on disk in dashboard.env and preview.env. A setup.env copy may remain in /opt/src/setup.env on the server.
  • The fallback commit email uses {login}@users.noreply.github.com. It should use the id-based form; store the GitHub user id at login.

🤖 Generated with Claude Code

Host-side repo_sync fetches into a root-owned bare cache with the token
passed via GIT_CONFIG_* env and ships git bundles into agents. Agent
remotes are plain URLs (startup scrub for existing containers), and all
delivery goes through createCommitOnBranch; auto_create_pr=false now only
suppresses the PR.
preview.sh fetches on the host into a root-only bare cache (token via
GIT_CONFIG_* env) and streams a bundle into the container, where the
preview user checks it out with a plain origin URL. /etc/preview-token
and the credential in .git/config are gone; 'preview scrub-tokens' (run
on deploy and before each update) removes them from existing previews.
hostSecrets now use exact-match lookup and refuse infrastructure keys.

Breaking: target repos' preview-config.nix must drop their git step;
old configs keep working only for public repos.
login sets a 10-minute HttpOnly/Secure/SameSite=Lax oauth_state cookie
scoped to the callback path and sends the same value as state; callback
rejects a missing or mismatched state (constant-time compare) and clears
the cookie. A cancelled GitHub authorization now redirects home instead
of failing query parsing.
Sessions carry a session_version that must match users.session_version,
and the role now comes from the DB on every request. Membership is
re-checked with the user's token after 10 min (2 min for member/admin
actions); losing membership or a revoked token bumps the version and
ends every session, while transient GitHub errors get a bounded grace
(1 h / 10 min) and never revoke. Role changes also bump the version.
check_org_membership maps 204/404/302/401 explicitly with redirects off.
The secrets key comes from SECRETS_ENCRYPTION_KEY, or else a 0600 key
file (default /var/lib/dashboard/secrets-key) generated atomically on
first start, so encryption is always on. Tokens live in
users.github_token_encrypted; a startup step encrypts legacy plaintext
tokens and NULLs them. Every read goes through auth::load_github_token,
and a token that won't decrypt asks the user to sign in again.
… 0.6)

Before polling or spawning, check that run_as_user is still an org
member with a usable token; if not, auto-disable the source with a
visible reason and a system audit event (transient errors just skip).
Without a label filter, only issues by OWNER/MEMBER/COLLABORATOR are
taken. Prompts are rebuilt at spawn with issue text inside per-prompt
random <untrusted_issue_*> fences, neutralized markers, and single-pass
template substitution.
Commits pushed through createCommitOnBranch carry Tekton-Task and
Co-authored-by trailers. PR bodies end with a footer naming the
requester (or the intake source and issue), a link to the task in the
dashboard, and the effective model, recorded per run in tasks.model
(OpenRouter only; NULL when the CLI default applies).
SECRETS_ENCRYPTION_KEY, JWT_SECRET and GITHUB_CLIENT_SECRET no longer
sit on persistent disk. server-config/dashboard-secrets.sh push fills
a template with op inject and installs it as
/run/dashboard-secrets/secrets.env, a required second EnvironmentFile
of the dashboard unit, only if all three values are non-empty; a
reboot leaves the dashboard down until they are pushed again. The
script also copies values to the clipboard and verifies by hash that
1Password holds the key the running dashboard uses, for the upgrade.

All three must be non-empty; the key-file fallback and the "not
configured" branches are gone. Config is read in a sync main, which
then removes the three from the environment so child processes don't
inherit them. On startup, before legacy tokens are encrypted, a sample
of the oldest stored ciphertext must decrypt or the dashboard refuses
to start.

deploy.sh checks the /run file, that dashboard.env and old key files
no longer hold secrets, and that the unit reads the file, then fails
if the dashboard isn't up after the restart. server-setup.sh writes the
/run file instead of dashboard.env and refuses to mint new secrets on
an existing install. make deps writes a random key into the local .env.
…#184 0.5)

A signed-in 1Password CLI session lets anything in that terminal read
every vault, so op is gone. dashboard-secrets.sh set opens a terminal
session to the server and reads the three values with read -s, one
paste each from the 1Password app; verify-key reads the key the same
way and compares hashes with the live one. Pasted CR/LF and blanks are
stripped, multi-line pastes are refused, and values are written
single-quoted, which systemd takes literally, so only a single quote,
whitespace and control characters are rejected. The template is gone;
the item is Tekton/key-tekton-00 by convention.
Prod's SECRETS_ENCRYPTION_KEY has an unknown format and can't change,
so set and verify-key now take any value without newlines or other
control characters. Values are written double-quoted with systemd's
EnvironmentFile escapes, and copy/check-file read them back with a
parser that follows env-file.c, so the process gets the pasted bytes.
Only a trailing CR and bracketed-paste markers are stripped, and echo
stays off for the whole prompt.

set compares the pasted key with the running dashboard's and asks
again on a mismatch, unless --new-key. check-live reports, without
values, whether the live values can be entered; copy-live reads
/proc/<pid>/environ NUL-safely and refuses multi-line values. An
interrupted set removes its temp file, and the preflight flags any
leftover one.
 0.2)

A repo's preview-config.nix was evaluated as root with --impure, so it
could read /run/dashboard-secrets, /var/secrets or any host file. Its
fixed-output builds could also reach loopback-trusted endpoints
(/internal/secrets, /internal/token, Caddy's admin API).

Evaluate it in restricted mode, as a systemd-run DynamicUser with no
network and the secret paths hidden. Restricted mode alone is not enough,
because getFlake "path:..." still copies readable host dirs into the
store. nixpkgs comes from /etc/nixos/flake.lock as a locked store path,
so closures are unchanged. A tekton-nixbld firewall chain keeps nixbld
builds off loopback and private/link-local addresses; preview.sh refuses
to build without it. Also pin swap off.
fedacking added a commit that referenced this pull request Nov 5, 2026
Webhook previews get a one-hour, read-only App token for the repo, but
dashboard previews still get the user's OAuth token (until #186). The doc
described only the first case. Raised by the Claude and Grok reviews.

Refs #190
fedacking added a commit that referenced this pull request Nov 5, 2026
* Mint per-repo GitHub App tokens and remove PAT auth from preview-webhook

getToken({repo, permissions, minTtlSeconds}) returns {token, expiresAt},
narrowed to one repo and the requested permissions, cached per
(repo, permissions). The repo owner must match the installation account
(case-insensitive). Startup fails if GITHUB_TOKEN is set, an App variable
is missing, or the key is unreadable or not an RSA PEM.

Callers are updated in the next commit; this one does not build alone.

Refs #181

* Request per-repo tokens in preview-webhook consumers

PR body edits ask for pull_requests:write on the PR's repo.
GET /internal/token now requires ?repo=owner/name (validated against a
strict pattern and a non-empty ALLOWED_REPOS), accepts ?min_ttl, returns a
contents:read token as {token, expires_at}, and keeps the localhost-only
check. The route moves to internal-token.ts so it can be tested.

Refs #181

* Add node:test suite for preview-webhook token provider and /internal/token

Covers per-(repo, permissions) minting and caching, refresh near expiry,
the min_ttl handoff after a build, the case-insensitive owner check,
startup failures (GITHUB_TOKEN set, missing App vars, bad key), and the
/internal/token validation. GitHub is a stubbed fetch; no network.

Tests compile to dist-test/ via tsconfig.test.json and run with npm test.

Refs #181

* Use per-repo App tokens in preview.sh and refresh before container handoff

get_github_token takes the repo and an optional minimum TTL and asks
/internal/token for a contents:read token for that repo. create and
update fetch a fresh token (at least 50 minutes left) right before
write_env_files and before the code-only update rewrites
/etc/preview-token and the git remote, since a long build can outlast
the token fetched beforehand.

load_secrets no longer saves and restores the caller's GITHUB_TOKEN; it
refuses to run if preview.env still sets one.

Refs #181

* Configure the GitHub App in setup scripts

setup.sh asks for (or reads from setup.env) the local path of the App's
private key and uploads it to /var/secrets/github-app.pem as root 0400.
server-setup.sh prompts for the App ID and installation ID, writes the
three GITHUB_APP_* variables to preview.env, and checks the key's
presence, owner and mode. setup.env.example documents the new variables.

Refs #181

* Preflight the GitHub App config in deploy.sh before installing anything

preflight_github_app runs right after sync_code, for every component. It
fails the deploy with migration steps if preview.env still sets
GITHUB_TOKEN, lacks a GITHUB_APP_* variable, or points the key path
elsewhere, or if /var/secrets/github-app.pem is missing or not root 0400.
It only inspects names, paths and modes, and never prints values.

Refs #181

* Document GitHub App setup and migration from the PAT

deployment-guide.md replaces the PAT section with App registration
(private, selected repos, contents:read + pull_requests:write, webhook
off), key rotation, the deploy preflight, and a "Migrating from a PAT"
section ending with revoking the PAT. preview-deployments.md lists the
App variables and explains the per-repo, short-lived tokens;
adding-a-new-service.md warns services not to rely on /etc/preview-token
after setup; architecture.md lists the token provider.

Refs #181

* Report a missing key path once in the GitHub App preflight

The path check also fired when GITHUB_APP_PRIVATE_KEY_PATH was missing,
repeating the "does not set" line. Found on staging.

Refs #190

* Say which token /etc/preview-token holds for dashboard previews

Webhook previews get a one-hour, read-only App token for the repo, but
dashboard previews still get the user's OAuth token (until #186). The doc
described only the first case. Raised by the Claude and Grok reviews.

Refs #190
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

GitHub identity Phase 0: stop user-token leaks

1 participant