Repository navigation
nix-eval never builds, so a broken Nix derivation can merge into v2 undetected #52863
Description
Description
Since #52143, nix-eval does run on v2 pull requests, but it only evaluates drvPath (.github/workflows/nix-eval.yml:52, :70, :84) and never builds. A derivation that evaluates cleanly and fails to build is therefore invisible until after merge.
Two details make it weaker than it looks:
desktopsits inOPTIONAL_PACKAGES(nix-eval.yml:43), and a failed desktop evaluation emits::warning::rather than an error (:74). A desktop build can be broken without turning the check red.nix-hashes.ymltriggers onpushtodev/beta/v2, neverpull_request, and it only recomputes.nodeModulesinnix/hashes.json. It cannot look at a PR, and it cannot catch a derivation bug at all.
#50408 is the concrete case: nix/opencode.nix kept calling opencode completion, which v2 dropped. That evaluated fine and broke the build. It was reported by a user eight days after landing.
Plugins
None
OpenCode version
v2 at eb1a9ea4d9
Steps to reproduce
- Land a change on a
v2PR that alters a build phase, for example replacing a CLI invocation innix/opencode.nixwith a flag that does not exist. nix-evalpasses: it only evaluates, anddesktopis warning-only anyway.nix-hashesdoes not run on the PR.- Merge, then run
nix build .#packages.x86_64-linux.opencodeonv2. It fails inpostInstall.
Screenshot and/or share link
Not applicable; reproduction is command line only.
Operating System
Linux x86_64, nixpkgs 26.11, nix 2.34.8
Terminal
Not applicable.
Building opencode in the workflow would catch the #50408 class. Alternatively, drop desktop from OPTIONAL_PACKAGES once #11755 is fixed so its evaluation failure fails the job like the others.
Happy to contribute whichever direction you prefer.
Activity
github-actions commentedon Oct 2, 2026
This issue might be a duplicate of existing issues. Please check:
nix-evalevaluates derivations without building them.jerome-benoit commentedon Oct 2, 2026
Checked both, and I think this is distinct:
.github/TEAM_MEMBERSmissing from the fileset), already fixed. It is an instance of this class, not the guardrail itself.What is still open here is the workflow configuration:
nix-evalevaluatingdrvPathonly,desktopbeing warning-only, andnix-hasheshaving nopull_requesttrigger.opencodejr commentedon Oct 27, 2026
Thanks for the detailed write-up, @jerome-benoit!
To restate the request: the Nix CI check should actually build the
opencodederivation on pull requests, not just evaluate itsdrvPath. That would catch build-phase breakage like #50408 before merge. A second option is to make thedesktopevaluation fail the job once #11755 is resolved.A note for whoever picks this up: on
devthe workflow is now.github/workflows/check-nix.yml(renamed in #59077) and it runs ondevpull requests. It still only runsnix eval ...drvPath,desktopis still warning-only, andbot-nix-hashes.ymlstill runs only on push todev. Thecompletioncall from #50408 has since been switched to--completionsinnix/opencode.nix.Related:
desktopevaluation fatalThe maintainers will consider this and which direction they prefer, including the CI time a full build would add. Thanks also for offering to contribute.