Add a curated llms.txt generated from the docs.json nav - #47
Conversation
|
Preview deployment for your docs. Learn more about Mintlify Previews.
💡 Tip: Enable Workflows to automatically generate PRs for you. |
| @@ -0,0 +1,151 @@ | |||
| #!/usr/bin/env bash | |||
There was a problem hiding this comment.
This makes sense ✅ My one real concern is that this needs a human to run a script, and I don't think we've priced that in.
The CI check finds drift, it doesn't fix it. That's fine when a person is editing docs by hand. It's less fine for sync-man-pages.yml, which runs on cron every morning and opens a bot PR. Descriptions come from the ## NAME line, and those get synced from flox/flox. So the day someone upstream rewords a NAME line, FloxBot's PR goes red and whoever's on rotation has to know the fix is "run the generator and push to the bot's branch." We'd be adding a manual step to a job that currently needs nobody.
Worth noting the check isn't enforced either. I get a 404 on branch protection for main, so a red check doesn't block anything right now (might just be my token lacking admin scope, worth a look in settings). check-man-nav.yml has the same gap so this isn't new, but "CI fails until regenerated" only does work if failing means something.
I've added a build script for the one on the website maybe we do something similar?
There was a problem hiding this comment.
Good catch, and you're right that it wasn't priced in. Fixed in 835d4f8: the sync job now regenerates llms.txt after sync-man-pages.sh, with llms.txt added to add-paths (without that, create-pull-request drops it from the commit). The job that causes the drift is the one that can fix it, so FloxBot's PR arrives correct and nobody on rotation needs to know the incantation. The step sits outside flox activate on purpose — that environment ships node, vale and pandoc, and the generator needs python3 from the runner.
Verified by rewording a ## NAME line locally: it propagates to exactly one llms.txt line, which the new step now carries into the bot's commit.
On the website build script — that pattern doesn't port, unfortunately. floxwebsite generates llms-full.txt in postbuild because Vercel runs its build. Mintlify builds this site itself from repo contents and gives us no arbitrary build hook, so the file has to be committed and the automation has to happen at commit time. Regenerating in the producing job is the closest equivalent we have.
On branch protection: it wasn't your token. main is protected by a ruleset, not classic protection, and /branches/main/protection 404s for everyone in that case. The active "Main" ruleset has deletion, non_fast_forward, merge_queue and pull_request (1 approval) — and no required_status_checks rule at all, so you're right that red blocks nothing. Two traps before we add one, both of which bite this repo specifically:
- Both checks use
paths:filters. A required check that gets skipped sits "expected" forever and blocks the PR. - The ruleset has a merge queue, and neither workflow has a
merge_group:trigger — a required check that never reports there times out at 60 minutes and ejects the entry.
So it's drop the path filters (both jobs run in seconds) plus add merge_group:, then require both checks. Happy to do that as its own PR since it affects check-man-nav too — it's a settings change and shouldn't ride along here.
Worth flagging: check-man-nav has the identical shape on the bot PR — a new command's page lands with no nav entry and goes red. That one I'd leave manual, since picking the group is a curation call, but we could auto-insert into "Other" so the reviewer re-groups rather than unblocks.
If hand-edited drift ever shows up in practice, the next step is a post-merge self-heal on main (regenerate, open a FloxBot PR on drift), which covers forks and web edits too. Not worth it until we see it.
billlevine
left a comment
billlevine
left a comment
There was a problem hiding this comment.
Peer panel review — 4 seats, 2 rounds, unanimous REQUEST-CHANGES
Four independent reviewers examined this diff blind (bare Codex; a Codex specialist wearing a senior CI/CD & generated-artifact-tooling role; Claude under a senior-code-reviewer harness; Claude under the Forge lens pipeline), then exchanged findings and reconciled. All four seats say the architecture is right — deriving the index from the nav rather than hand-curating it is the correct call, and I confirmed the generator is deterministic and idempotent, with the committed llms.txt byte-identical to its output on this head. Every blocker below is in the enforcement layer or in branch state, not in the idea.
First, a note on the existing review: @Kaysahni's blocking concern is genuinely addressed by 835d4f8. The panel re-derived that independently — the sync job now regenerates llms.txt and carries it in add-paths, so FloxBot's PR arrives correct with no human incantation. The formal "changes requested" state just has not been re-requested, which is a human step. The finding below about that same step is a different failure mode, not a reopening.
The two things that block merge.
The branch is 22 days stale, and the drift gate this PR introduces does not pass on this PR's own merge result. git merge-tree --write-tree origin/main HEAD merges cleanly; regenerating on that tree yields 103 pages in 21 sections against the committed file's 99 in 20 — adding concepts/authentication, concepts/service-accounts, concepts/personal-access-tokens and a whole ## Changelog section, and dropping the two retired man pages. Production corroborates it: of the 105 URLs in the committed llms.txt, exactly two 404, and they are precisely man/nix-builds.toml.md and man/flox-build-update-catalogs.md. The green check-llms-txt on this PR was created 2026-07-28 and predates every relevant base commit; actions/checkout does test the merge ref, but GitHub does not re-run a PR's checks when the base moves, so that verdict is stale rather than absent. Refresh the branch and regenerate — rebase-or-merge is your call, so I have not touched it.
The other is the new regenerate step in sync-man-pages.yml, and main's own history is the evidence. 14de796 (2026-07-29) deleted those same two man pages, their nav entries remained, both sidebar links 404ed, and 28a1a9a (2026-08-06, "remove retired man pages from nav, catch dangling entries in CI") is a human cleaning it up. That is main's deliberate design: the bot opens the PR and the PR's CI fails so a person fixes the nav — note sync-man-pages.sh:94-96 calls check-man-nav.sh with || echo warning specifically to keep it non-blocking inside the sync. The new unconditional step inverts it: the next retirement makes the job exit 1 before create-pull-request, so no PR appears at all and the man pages quietly stop updating.
What the panel converged on beyond those: nothing enforces the "0 bare links" property the PR is sold on (the description-less branch is a normal successful outcome, and described is printed but never compared to total); the generator writes its output before it validates and prints a success line on the failing path; walk() recognizes three container keys where its own sibling check-man-nav.sh recurses every value; the stated invariant "a page cannot reach the site without also reaching llms.txt" is already false in the tree (concepts/flox-vs-containers-faq.mdx is live, linked from an indexed page, in no nav, in no index, and no check fires); and four accuracy problems in the preamble, which is the one file whose entire purpose is to give agents a correct mental model.
Fixups pushed. I have pushed the low-risk, converged half as separate commits and replied on the threads they touch — preamble accuracy (flox edit, the flox activate mechanism, WSL2, the generations glossary, the Codex install path), generator robustness (validate-before-write, explicit UTF-8), two title-echo descriptions, an AGENTS.md section for the new generated artifact, and the bot PR body wording. I deliberately left alone everything with a real design tradeoff, everything touching CI trigger semantics — you scoped the path-filter and required-checks work to its own PR on this thread already — and the branch refresh.
Beyond this PR. update-flox-version.yml seds **/*.mdx and opens a PR with no regenerate step, so it is the same drift source as its sibling, violating the principle this PR's own comment states. Latent today — no X.Y.Z appears in any title: or description:. Separately, llms.txt.header sits outside Vale (.vale.ini scopes to [*.mdx]), so the most agent-facing prose in the repo is the only prose not linted. Both are worth issues rather than changes here.
The ## OpenAPI Specs gap is correctly excluded — there is no openapi reference anywhere in the repo, so a nav-derived generator cannot produce it. DEV-203 is the right home.
(Peer panel: codex · codex-specialist · claude · forge. 4 seats, 2 rounds plus an evidence round. Weigh the two Codex seats as one vendor. Full artifacts: ~/.peer-panel/runs/docs-pr47.)
| # causes the drift is the one that can fix it without a human. | ||
| # Deliberately outside `flox activate` — this environment ships node, | ||
| # vale and pandoc, and the generator needs python3 from the runner. | ||
| - name: Regenerate llms.txt |
There was a problem hiding this comment.
blocking: The next time a command is retired upstream, the daily man-page sync will stop producing PRs altogether instead of surfacing the retirement. sync-man-pages.sh:33 deletes man/*.mdx before regenerating, so a retired command leaves the hand-maintained docs.json pointing at nothing; this step then exits 1 at generate-llms-txt.sh:146-150 with no continue-on-error, before create-pull-request ever runs. This is not hypothetical — 14de796 did exactly that on 2026-07-29, and 28a1a9a ("catch dangling entries in CI") is a human cleaning it up a week later, which is the workflow this replaces.
Suggested: add continue-on-error: true to this step, so a nav that has gone dangling degrades to a red check on a PR a human can fix rather than to no PR at all. Note the remedy cannot be "let the bot fix the nav": add-paths is man + llms.txt and does not include docs.json. The neighbouring sync-man-pages.sh:94-96 already makes the same call the other way, invoking check-man-nav.sh with || echo warning.
(Peer panel: all four seats converged; the strongest finding on the panel.)
| @@ -0,0 +1,221 @@ | |||
| # Flox | |||
There was a problem hiding this comment.
blocking: Merging as-is lands an llms.txt that its own new gate rejects. git merge-tree --write-tree origin/main HEAD merges cleanly, and regenerating on that tree gives 103 pages in 21 sections against this file's 99 in 20 — it adds concepts/authentication, concepts/service-accounts, concepts/personal-access-tokens and a ## Changelog section, and drops man/nix-builds.toml and man/flox-build-update-catalogs, which 14de796 retired. Production already shows the tail of it: of the 105 URLs here, exactly those two 404. The green check-llms-txt run was created 2026-07-28 and predates every one of 14de796, 278d6c9, 704c7d9, 24928be, a7016c7.
Suggested: bring the branch up to date with main and re-run ./scripts/generate-llms-txt.sh. I left this one alone deliberately — rebase versus merge is a branch-shape decision that is yours, not a reviewer's.
(Peer panel: established in the evidence round; all four seats treated it as ground truth.)
There was a problem hiding this comment.
Update: CI has now confirmed this, and my fixup push is what surfaced it.
Pushing the three fixup commits re-ran check-llms-txt against the current base for the first time since 2026-07-28, and it fails — run 32283426691. The reported drift is line-for-line what the evidence round predicted, and contains none of my changes:
+- [Authentication](.../concepts/authentication.md): ...
+- [Service accounts](.../concepts/service-accounts.md): ...
+- [Personal access tokens](.../concepts/personal-access-tokens.md): ...
-- [nix-builds.toml](.../man/nix-builds.toml.md): ...
-- [flox build update catalogs](.../man/flox-build-update-catalogs.md): ...
+## Changelog
+- [2026] / [2025] / [2024] ...
To be unambiguous about cause: this is not a regression from the fixups. It is the pre-existing base drift, and it was always going to appear on the next run — the previous green was a verdict about a base from three weeks ago. My push just triggered the re-run. Everything my commits touched is already reflected in the regenerated llms.txt, which is why none of it appears above.
This also incidentally demonstrates the point on the on: thread: the gate works, it simply had no reason to re-evaluate until something pushed.
Still leaving the remedy to you, since it is a branch-shape call and a rebase would need a force-push:
git fetch origin && git merge origin/main # or rebase, your preference
./scripts/generate-llms-txt.sh
git commit -am "Regenerate llms.txt against current main"I confirmed that resolves it: applying the generator to git merge-tree --write-tree origin/main HEAD produces exactly the file the check is asking for.
| text = open(path).read() | ||
| fm = frontmatter(text) | ||
| label = fm.get("title") or os.path.basename(page) | ||
| desc = fm.get("description") or name_line(text) |
There was a problem hiding this comment.
non-blocking: A page added later can pass the drift check while putting bare links back into the file this PR exists to de-bare. desc = fm.get("description") or name_line(text) has no failure branch — when both come back empty, line 134's ternary emits - [Title](url) as a normal successful outcome, and described is counted and printed but never compared against total. "0 of 99" is a property of today's content rather than an invariant the generator holds. The title half is safe, since line 129 falls back to the basename.
Suggested: collect description-less pages the way missing already collects absent .mdx files, and exit non-zero with the same shape of message. That one condition also closes the folded-scalar case, where description: > yields the literal > and is counted as described.
(Peer panel: found independently by three seats before the exchange.)
There was a problem hiding this comment.
Fixed in 36b60a8. undescribed now collects alongside missing and fails the run with the same shape of message, before anything is written — so "0 bare links" is an invariant the generator holds rather than a property of today's content.
The folded-scalar case is folded into the same condition as you suggested, but one step earlier: frontmatter() now drops a value matching ^[>|][+-]?\d*$ instead of returning it, since a block-scalar indicator opens a value this one-line parser cannot read at all. It therefore reaches the new check as absent rather than as the literal >.
Both reproduced against the pre-fix script first, to confirm I was fixing what you described:
# nav page with no description: frontmatter and no ## NAME line
OLD: wrote ...: 104 pages in 21 sections, 103 with descriptions (exit 0)
- [Scratch page](https://flox.dev/docs/concepts/scratch-nodesc.md)
NEW: error: nav pages with no description:
concepts/scratch-nodesc (exit 1)
# description: >
OLD: wrote ...: 104 pages in 21 sections, 104 with descriptions (exit 0)
- [Folded](https://flox.dev/docs/concepts/scratch-nodesc.md): >
NEW: error: nav pages with no description: (exit 1)
Also confirmed the failing run leaves the committed llms.txt untouched, since the new check sits with missing above the write.
llms.txt is byte-identical after the change — still 103 pages, all with descriptions.
| crumbs = crumbs if node["tab"] in DROP_TABS else crumbs + [node["tab"]] | ||
| elif "group" in node: | ||
| crumbs = crumbs + [node["group"]] | ||
| for key in ("tabs", "groups", "pages"): |
There was a problem hiding this comment.
non-blocking: A future nav restructure can shrink the published index with no error and no CI failure, because this walker and its sibling disagree about what the nav is. for key in ("tabs", "groups", "pages") skips any other Mintlify container — anchors, dropdowns, versions, languages — while check-man-nav.sh:41-49 recurses for value in node.values(), i.e. everything. Seats reproduced a page under an injected anchors node vanishing with exit 0 and no warning, while check-man-nav.sh printed ok against the same docs.json. Latent: the live nav uses only the three keys you handle.
Suggested: recurse every dict value, matching the sibling — or raise on an unrecognized container key, so the failure is loud like every other one in this script.
(Peer panel: all four seats converged; held at Minor on severity because nothing triggers it today.)
There was a problem hiding this comment.
Fixed in 36b60a8, taking your second option — raise on an unrecognized container key rather than recurse every dict value.
The reason for that half of the choice: matching check-man-nav.sh exactly does not port. Its walker collects every string in the tree and that is safe there, because it only ever compares against man/<name> and a group or tab label cannot collide with one. This walker has to tell a page path from a section label, so recursing node.values() indiscriminately would start emitting "Setup" and "rocket" as pages. Descending an unknown container also means inventing its section semantics — what an anchor or a dropdown contributes to an H2 title — and I would rather not guess that for structures this repo does not use.
So CONTAINER_KEYS is now explicit, and any other key whose value is a list or dict fails the run and names itself. Scalar metadata (icon, href, root) is untouched, so this does not fire on anything in the nav today.
Reproduced with your injected anchors node:
OLD: wrote ...: 103 pages in 21 sections (exit 0)
grep -c flox-vs-containers-faq llms.txt -> 0 # silently dropped
NEW: error: unhandled docs.json nav container(s): anchors — teach walk()
how to descend them (and what they contribute to a section title)
before adding them to the nav (exit 1)
The tradeoff is explicit: adding anchors to the nav now costs a code change here. That seems the right way round, given the alternative was a shorter published index with a green check.
|
|
||
| - name: Check for drift | ||
| run: | | ||
| if ! git diff --exit-code --stat llms.txt; then |
There was a problem hiding this comment.
non-blocking: A PR that deletes llms.txt passes this check green — the one outcome the workflow exists to prevent. The regenerate step recreates the file as untracked, and git diff cannot see untracked files, so the diff comes back empty. I reproduced it: commit the file, git rm it, recreate it, run this exact condition — no drift reported. Merging such a PR hands /docs/llms.txt back to Mintlify's generated version with CI green.
Suggested: git add -N llms.txt before the diff, or git add llms.txt && git diff --cached --exit-code. I left this one for you rather than pushing it, since it sits in the same CI rework you scoped to a separate PR on this thread.
(Peer panel: found independently by two seats, each with its own reproduction.)
There was a problem hiding this comment.
Fixed in 49cc60c. You are right that I parked this with the CI rework, and that was the wrong bucket — the path filters and required checks are a settings call, this is the check's own logic being wrong. It should not have waited on a separate PR.
Took the second of your two forms. I tried git add --intent-to-add first and it does not work on git 2.51: git status shows A llms.txt, but git diff --cached --stat llms.txt prints nothing and exits 0, so the deleted case still passed green. A real git add plus git diff --cached does catch it.
Verified all three outcomes against a scratch repo, since a check that only catches deletion would be a worse bug than the one it replaced:
in-sync -> exit 0 (green, as before)
committed file stale -> exit 1
committed file deleted -> exit 1 (was exit 0)
Your reproduction was exact, including that the recreated file being untracked is what hid it.
| on: | ||
| pull_request: | ||
| paths: | ||
| - '**/*.mdx' |
There was a problem hiding this comment.
suggestion: '**/*.mdx' likely does not match the two root-level pages, and both feed this file — index.mdx and flox-5-minutes.mdx are nav pages emitted at llms.txt:66 and :77, so editing either one's title: or description: moves the generated output. GitHub documents ** as matching zero or more of any character including /, and its own idiom for extension-matching across a repo is '**.js', not '**/*.js'; I could not retrieve the filter-pattern cheat sheet to settle the **/ case outright. In the drifted case llms.txt is precisely what the PR did not update, so the sibling 'llms.txt' path entry does not rescue it.
Suggested: '**.mdx' — correct under either reading, one character. Left unpushed: this is inside the path-filter and required-checks rework you already scoped to its own PR on this thread.
(Peer panel: two seats; the evidence round settled the remedy but not the underlying glob semantics.)
There was a problem hiding this comment.
Fixed in 49cc60c — '**.mdx', as suggested.
Same correction as on the sibling thread: I had put this in the path-filter rework, but that PR's plan is to drop both filters, and until it lands this pattern is live and wrong. One character is not worth blocking on a settings change.
On the glob semantics you could not settle: I got as far as GitHub's paths documentation, which gives '**.js' as the pattern for "anytime you push a JavaScript file" — its own idiom for extension-matching across a repo, exactly as you said. I could not retrieve the cheat-sheet rows either, so I have not proved the **/ case outright. Taking the change on your argument that it is correct under both readings rather than on a resolution of the underlying question.
For the record on why it matters here: index.mdx and flox-5-minutes.mdx are emitted at llms.txt:66 and :77, and in the drifted case llms.txt is precisely the file the PR did not touch — so the sibling 'llms.txt' entry cannot rescue it.
| # and committed, because Mintlify serves it from the repo rather than building | ||
| # it. Fail PRs where the committed file no longer matches what the generator | ||
| # produces — otherwise adding a page silently leaves it out of the index. | ||
| on: |
There was a problem hiding this comment.
question: With pull_request and workflow_dispatch only, a green verdict here can go stale rather than being re-derived — actions/checkout does check out the merge ref, so this job does test the merged tree, but GitHub does not re-run a PR's checks when the base moves. This PR is the demonstration: green from 2026-07-28, against a base that has changed five times since.
Suggested: requiring branches be up to date before merge (or a merge_group trigger) is what actually prevents this; a push: branches: [main] trigger only reports it afterwards. Both interact with the path filters and the merge queue exactly as you laid out on this thread — if that separate PR is where this belongs, this is just a note that the panel reached the same place independently.
(Peer panel: three seats proposed the push trigger; a fourth corrected them on the merge-ref point during reconciliation, and I ruled for it.)
There was a problem hiding this comment.
Leaving the on: block as-is, and agreeing with the fourth seat's correction: actions/checkout does check out the merge ref, so this job already tests the merged tree. The gap is purely that GitHub does not re-derive a verdict when the base moves, and no trigger fixes that. push: branches: [main] would report the drift after it merged, which is a different and weaker thing than preventing it — I do not want a trigger here whose only effect is to tell us we were already wrong.
The remedies that actually work are both outside this file: require branches be up to date before merge, or add a merge_group: trigger so the queue re-derives it. Those are the same ruleset change I scoped to its own PR upstream on this review — the one that has to drop the path filters and add merge_group: to both workflows before either check can be required, or a skipped check sits "expected" forever and a check that never reports to the queue times out at 60 minutes and ejects the entry. I would rather that landed as one coherent change than have a partial version of it ride along here.
Note that the two sibling findings on this workflow did not stay in that bucket — the drift check's blindness to a deleted llms.txt and the '**/*.mdx' glob are both fixed in 49cc60c. Those are this check's own logic being wrong; this one is a repository setting.
Your demonstration held, and this PR closed it rather than illustrating it further: the branch is now rebased onto current main with llms.txt regenerated as part of the rebase, and check-llms-txt is green against a base from today rather than from 2026-07-28.
|
Fixups from the peer panel are pushed — three commits on top of 835d4f8, each replied to on the thread it addresses. 30c4a32 — preamble accuracy. Four claims in 2631e35 — generator robustness. Validate before writing, so a run that exits 1 no longer leaves a truncated 742b57c — documentation. An The generator is still deterministic and idempotent after all of this, and still emits 99 pages / 20 sections / 99 descriptions. Deliberately not touched, all of it explained on the relevant thread: the branch refresh (rebase-vs-merge is yours), the sync-job abort on an upstream retirement (the remedy is constrained by (Peer panel fixup pass. Review: #pullrequestreview-4975071687) |
Mintlify auto-hosts an llms.txt, but it is a single flat alphabetical list with no preamble, and 40 of its 100 links carry no description at all because man/*.mdx has no `description:` frontmatter. Committing an llms.txt at the repo root overrides it. Rather than hand-maintain a curated file, generate it from the nav. docs.json is already where curation happens, and check-man-nav.sh already fails PRs that add a page without a nav entry, so a page cannot reach the site without also reaching llms.txt. Sections come from nav groups; descriptions come from frontmatter, falling back to the man page's `## NAME` line, which is the canonical one-liner shipped with each command. Result: 99 pages, 20 sections, 0 missing descriptions. The preamble in llms.txt.header restores agent guardrails that were lost when the docs moved off MkDocs — the non-interactive `flox activate` forms, `flox install` over the system package manager, editing manifest.toml directly, and the negation for the hallucinated `install.flox.dev` / `flox.dev/install` URLs. It sits ahead of the first H2 on purpose: strict llms.txt parsers extract only link lists from H2 sections, so prose under a heading gets dropped. Also drops the flox MCP server, which is deprecated in favour of the skill library, and repoints flox-agentic at flox-skills. The generated file omits Mintlify's `## OpenAPI Specs` section, which links a spec that currently serves Mintlify's plant-store sample (DEV-203). Restore it here once that is fixed. Refs AI-500
llms.txt takes each man page's description from its `## NAME` line, and those lines are synced from flox/flox. So the morning a command's NAME is reworded upstream, FloxBot's sync PR goes red on check-llms-txt and whoever is on rotation has to know the fix is "run the generator and push to the bot's branch" — a manual step added to a job that needs nobody today. Regenerate in the sync job instead. The job that causes the drift is the one that can fix it without a human, so the bot's PR arrives correct. `llms.txt` joins `add-paths`, without which create-pull-request drops the regenerated file from the commit. The step sits outside `flox activate` deliberately: that environment ships node, vale and pandoc, and the generator needs python3, which comes from the runner. check-llms-txt.yml stays as the net for hand-edited PRs, where the author is in a local checkout and the error names the command to run. Refs AI-500 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Peer review found four claims in the preamble that the docs it indexes contradict. This file exists to stop agents guessing, so a wrong mental model in it is worse than no entry. - `flox edit`: the old bullet told agents to edit `.flox/env/manifest.toml` directly, which per man/flox-edit.mdx blocks `flox install`, `uninstall`, `edit` and `upgrade` on any FloxHub-linked environment until `--sync` or `--reset`. Name `flox edit -f`, the documented non-interactive form, and state the sync caveat. - `flox activate`: the mode is selected by the invoking shell, not by the absence of a command argument. From a non-interactive shell with stdout redirected, bare `flox activate` is in-place mode, not a hang. The prescriptive half (`-c` / `--`) was already right and is unchanged. - WSL 2: install-flox/install.mdx marks support experimental and notes installation fails on WSL 1; carry both across. - Generations: there is no generations diff command — `ls man/ | grep generations` returns only history, list, rollback, switch. Also drop the clone requirement from the Codex install path. `codex plugin marketplace add` accepts `owner/repo`, so the remote form works and mirrors the Claude Code path; neither the header nor ide-extensions.mdx ever gave the `git clone` step, and in llms.txt an agent would run `marketplace add .` in whatever directory it happened to be standing in. Two descriptions restated their own link text; replace them with the same class of fix this PR already applied to flox-vs-containers and organizations. Regenerate llms.txt. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Two generator robustness fixes, both converged across all four reviewers. Validate first. The output was written at the top of the file and the `missing` check ran afterwards, so a run that exited 1 still replaced the committed llms.txt with a version missing the unresolvable pages — and printed `wrote ...: N pages` on its way out. Since the file is committed, a distracted `git add -A` could stage the truncated result, which would then pass the drift check on the next run. Moving the check above the write costs nothing and makes the failure leave no residue. Verified: with man/flox-gc.mdx removed the run now exits 1 with llms.txt byte-identical. The page count no longer subtracts `missing`, because reaching the write means there is nothing to subtract. Explicit encoding. All four `open()` calls used the ambient locale while llms.txt.header contains em dashes, so `LC_ALL=C ./scripts/generate-llms-txt.sh` died with UnicodeDecodeError. Worse than the crash: a latin-1 locale would have succeeded while writing different bytes, producing drift a contributor could not reproduce. CI is unaffected — GitHub runners are UTF-8 — so this is purely a local-run fix. Verified under LC_ALL=C LANG=C PYTHONUTF8=0. The two reads become `with` blocks, which also closes the handles they leaked. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
AGENTS.md already has a "Man pages are generated — do not edit directly" section naming the script, its dependency and the enforcing check. This PR adds a second generated-and-committed artifact with the same shape and left that documented nowhere: `grep -i llms AGENTS.md README.md CONTRIBUTING.md` returned nothing. A contributor's first contact with the new gate would have been a red check with no in-repo guidance, and anyone hand-editing llms.txt would have lost the work silently. Add the parallel section. The sync bot's PR body said any llms.txt change there "follows from a reworded `## NAME` line". Added, removed and renamed pages move it too — the link text comes from the frontmatter title the sync derives from the filename — so a reviewer told to expect only NAME rewordings would under-inspect a page-set change. Widen it. Add llms.txt.header to .mintignore. It is a build input rather than a page. Whether Mintlify would serve it is genuinely unresolved — it is not deployed yet, and Mintlify's llms.txt override is a named feature rather than evidence that arbitrary root files are served — but the line costs nothing and forecloses the question either way. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
742b57c to
4e1a1f4
Compare
Three things the peer panel found, all of the same shape: a property the PR claims held for today's content and would have stopped holding silently. "0 bare links" — the reason this file overrides Mintlify's, which leaves 40 of 100 bare — had no failure branch. A page with neither `description:` frontmatter nor a `## NAME` line emitted `- [Title](url)` as a normal successful outcome, and the drift check could not catch it, since it only compares the committed file against a fresh run. Bare links are now collected like `missing` already is and fail the run before it writes. A `description: >` block scalar is folded into the same condition: this one-line frontmatter parser cannot read the continuation lines, and was emitting the literal `>` as the description while counting the page as described. walk() descended `tabs`, `groups` and `pages` and skipped every other Mintlify container — `anchors`, `dropdowns`, `versions`, `languages` — so a nav restructure could shrink the published index with exit 0, while check-man-nav.sh (which recurses every dict value) kept printing `ok` against the same docs.json. Reproduced by injecting an `anchors` node: the page under it vanished with no error. It now refuses a nav it does not fully understand, rather than guessing at section semantics for containers this repo does not use. And the header comment's guarantee stops at man/: check-man-nav.sh iterates `man/*.mdx` only, so an ordinary page can still ship out of the nav and out of llms.txt. concepts/flox-vs-containers-faq.mdx is live, linked from concepts/flox-vs-containers.mdx:18, and in neither. llms.txt is byte-identical after all three — none of this changes today's output.
A PR that DELETES llms.txt passed green — the one outcome this workflow exists to prevent. The regenerate step recreates the file as untracked, `git diff` cannot see untracked files, so the drift came back empty and the merge would hand /docs/llms.txt back to Mintlify's generated version. Reproduced end to end: commit the file, `git rm` it, regenerate, run the old condition — exit 0. Staging first and diffing the index catches it. Verified across all three cases: in-sync exits 0, a stale committed file exits 1, a deleted one exits 1. `'**/*.mdx'` does not match the two root-level nav pages. GitHub documents `**` as matching zero or more of any character including `/`, which makes the `/` in `'**/*.mdx'` a literal one, and its own idiom for "any .js file anywhere" is `'**.js'`. index.mdx and flox-5-minutes.mdx are nav pages emitted into llms.txt, so editing either one's title or description moves the generated file — and in exactly that case llms.txt is what the PR did not touch, so the sibling `'llms.txt'` entry does not rescue it. Neither of these is the path-filter and required-checks rework scoped to its own PR; both are this check's own logic.
3 participants
Publishes a curated
llms.txtat/docs/llms.txt, generated from thedocs.jsonnavigation tree so it cannot drift.Part of the llms.txt reconciliation tracked in AI-500. A companion PR updates the site-level file in
flox/floxwebsite.Why
Mintlify auto-hosts an
llms.txt. It is one flat alphabetical list with no preamble, and 40 of its 100 links carry no description at all — every man page, plusflox-vs-containersandorganizations— becauseman/*.mdxhas nodescription:frontmatter. Committing anllms.txtat the repo root overrides it.Separately, the MkDocs→Mintlify migration dropped a set of agent guardrails that used to ship in floxdocs'
llms.txtpreamble and now exist nowhere on the docs site.Approach: generate, don't hand-maintain
A hand-curated file drifts the moment someone adds a page. This derives it from the nav instead.
docs.jsonis already where curation happens — the CLI reference is already grouped into Overview / Setup / Environments / Packages / Sharing / Build & Publish / Services / Generations / Other — andcheck-man-nav.shalready fails PRs that add a page without a nav entry. So a page cannot reach the site without also reachingllms.txt. Curation happens once, in the place it already happens.Docs,OpenAPI Specs)/llms.txtWhat ships
scripts/generate-llms-txt.sh— walks thedocs.jsonnav, emits one H2 per group in nav order. Link text is the page's frontmattertitle; the description is its frontmatterdescription, falling back to the man page's## NAMEline, which is the canonical one-liner shipped with each command. Exits non-zero if the nav references a page with no.mdx.llms.txt.header— the hand-written preamble, copied verbatim ahead of the generated sections.llms.txt— generated and committed, since Mintlify serves it from the repo rather than building it..github/workflows/check-llms-txt.yml— regenerates and fails on drift. Mirrorscheck-man-nav.yml.concepts/flox-vs-containers.mdxandconcepts/organizations.mdx, the only two non-man nav pages that lacked one. These also improve search and the sidebar.Restored agent guardrails
These shipped in floxdocs and were lost in the migration:
flox activateforms. Bareflox activatelaunches an interactive sub-shell that hangs a non-interactive harness;-cruns with hooks and profile scripts,--is exec mode and skips profile scripts.flox install, notapt/yum/dnf/brew..flox/env/manifest.tomldirectly rather thanflox edit, which opens\$EDITOR.flox search/flox show.curl | bashinstaller;install.flox.devandflox.dev/installdo not exist. This one had its own bugfix commit in floxdocs (e3c2bf6), so it was a response to observed hallucination.Note the old floxdocs rule was a blanket "NEVER run `flox activate` interactively." Reading
flox-activate.mdx, interactive is one of four documented, legitimate modes — so this scopes the rule to non-interactive harnesses instead of contradicting the man page or the site-levelllms.txt.The preamble sits ahead of the first H2 deliberately. Strict llms.txt parsers extract only link lists from H2 sections, so prose under a heading gets dropped. Same choice floxwebsite made.
MCP server removed
The flox MCP server is deprecated in favour of the skill library, so it is gone from the preamble and from
install-flox/ide-extensions.mdx. That page's "Skills and MCP" tab is now "Agent skills", repointed fromflox/flox-agenticto flox/flox-skills (pushed 2026-07-27 vs flox-agentic's 2026-07-02), covering thefloxandfloxifyskills with Claude Code, Codex, and skills.sh install paths.No references to
mcporflox-agenticremain anywhere in the repo.Known gap
The generated file omits Mintlify's
## OpenAPI Specssection. That link resolves to"title": "OpenAPI Plant Store"served againstsandbox.mintlify.com— Mintlify's sample spec, tracked in DEV-203. Restore the section here once that is fixed.Verification
llms.txtreturn 200grep -ri 'mcp\|flox-agentic'over the repo returns nothing🤖 Generated with Claude Code