fix: repair master — lint, a dropped method, vendored drift, scorecard - #132
fix: repair master — lint, a dropped method, vendored drift, scorecard#132Kartikey1306 wants to merge 3 commits into
Conversation
CI -- ebuild has been red on master since the 09-08 batch merge, and the first failing step (ruff) has hidden the ones behind it. Lint (ruff, all nine Test legs): - test_build_dir_resolution.py imported shutil twice (F811). - test_package_recipe.py lost its trailing newline (W292). - test_ci_gate.py had `import itertools` / `import re` two hundred lines down (E402) -- my own embeddedos-org#103, replayed onto a file that had moved. These three hunks are byte-identical to embeddedos-org#122's, so either PR merging first leaves the other clean. Type check and tests (never reached on master since 09-08): - ebuild/packages/index_sync.py calls PackageRecipe.to_dict(), which embeddedos-org#111 defined and embeddedos-org#112 -- merged five minutes later from a base without it -- deleted in its replay. mypy names it once; pytest fails nine test_index_sync cases with AttributeError. The method is restored verbatim from embeddedos-org#111 (cc90078): it emits the `package:`/`build:` keys parse_recipe() reads back, which an asdict() replacement would not. Vendored core drift: - embeddedos-org#109 (dba3d83) edited core/eos/docs/three-way-alignment.md, a vendored copy pinned to eos 5544c98, so drift went 44 -> 45 and the guard failed as designed. Reverted to the pinned content (blob 7f9c8c1, the same bytes as eos:docs/three-way-alignment.md at the pin). The alignment note belongs in ebuild's own docs or upstream in eos, not in the snapshot. OSSF Scorecard: - ossf/scorecard-action@v2.4.0 pulls gcr.io/openssf/scorecard-action, and gcr.io now refuses the pull ("requires billing to be enabled"). v2.4.3 pulls from ghcr.io; eos already pins it and its Scorecard job is green. Not in this PR: EoSim Sanity's Windows/macOS legs install a wheel that has never been published; embeddedos-org#121 (srpatcha) already replaces that with the clone the other legs use. Verified locally: ruff clean, yamllint clean, mypy clean over 107 files, 680 passed / 1 skipped, scripts/check_vendor_drift.py 44/44 and 46/46.
|
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
…/3.11, yamllint on Windows Both surfaced on this branch's first CI run, once ruff let the job get past its first step. - ebuild/plugins/__init__.py: on Python 3.10 and 3.11 the stubs type entry_points() as the deprecated mapping, and its .get() wants an EntryPoints default, so mypy fails with arg-type. The line carried a '# type: ignore[attr-defined]' -- the wrong error code, so it suppressed nothing. Spelled out with a cast, byte-identical to embeddedos-org#122's hunk (54605f0). - .yamllint.yml: the Windows runners check out with core.autocrlf=true, so every YAML file arrives as CRLF and the default new-lines: unix rule rejected every line. The step was added on 09-03 and had never passed on that leg. new-lines: platform accepts the checkout's own convention.
…e checkout's line ending new-lines: platform was the wrong fix. The Windows runners' autocrlf turns LF files into CRLF -- except a file that already carries a stray CR, which git leaves alone, and auto-assign.yml had one on its last line. So under 'platform' Windows expected CRLF and got LF on that file's first line, and the leg was red again for the opposite reason. Pin *.yml and *.yaml to eol=lf so every OS lints the same bytes, keep yamllint's default unix rule, and drop the stray CR.
srpatcha
left a comment
There was a problem hiding this comment.
Thank you for the forensic write-up — it made this easy to verify. I re-ran each check on the branch: ruff clean, mypy loses exactly the two code errors (plugins/__init__.py:46, index_sync.py:354), pytest 674 passed with only the environment-only e2e build test failing, and check_vendor_drift.py is back to 44/44 where master fails at 45. The three-way-alignment.md blob hash matches the pinned 7f9c8c1, and I confirmed no other YAML file carried a CR before the .gitattributes pin.
Two optional follow-ups on the restored method, neither blocking:
ebuild/packages/recipe.py:92-114—to_dict()predatesinstall_args, so that field is dropped on round-trip throughindex_sync. Addingif self.install_args: data["install_args"] = list(self.install_args)would make it complete, and wrapping the other lists inlist(...)avoids handing out live references.-
- Since this method was already lost once in a replay, a tiny test asserting
parse_recipe(r.to_dict()) == rwould make the next disappearance a red test rather than nineAttributeErrors.
Approving — this should land first in ebuild. For the maintainers: this supersedes #122 (identical hunks) and #119/#124 (to_dict), and it will conflict with #127's unrelatedrecipe.py/pluginshunks.
- Since this method was already lost once in a replay, a tiny test asserting
srpatcha
left a comment
There was a problem hiding this comment.
Review — ebuild#132 "fix: repair master — lint, a dropped method, vendored drift, scorecard"
head: f8cc357 author: Kartikey1306 ci: pass (30 checks green, Create GitHub Release skipped)
Verdict: The most complete diagnosis of master's red state in the current queue, and the only PR in this batch with a full green matrix behind it — nine Test legs, CodeQL, vendor drift, eleven EoSim targets. Every claim in the body that I could check independently held. One defect: the restored PackageRecipe.to_dict() does not emit install_args, so it is not the round-tripping serializer the body says it is. That is latent rather than live today, and the reasoning for why is below.
Findings
| # | Severity | File:line | Finding | Recommended fix |
|---|---|---|---|---|
| 1 | Medium | ebuild/packages/recipe.py, to_dict() |
install_args is dropped. parse_recipe() reads ten optional fields; to_dict() emits nine. Verified: a recipe with install_args=["DESTDIR=/stage","--strip"] serializes to ['build','dependencies','package','patches','url','version'] and reconstructs with install_args == [] — parse_recipe(yaml.safe_load(yaml.safe_dump(r.to_dict()))) != r. This matters because the write and the read are two halves of one loop inside the product: index_sync.py:354 writes these files into recipes_dir, get_recipe_dirs() at :122 hands that directory to RecipeRegistry, and registry.py:114 loads them back through load_recipe(). No data is lost today, because the only producer — the entry→dict mapping at index_sync.py:335-347 — never populates install_args either, so the field is always empty by the time to_dict() sees it. The defect is that both halves silently agree to drop a field the schema supports, and the first recipe that carries one loses it with no error. The body's claim that this "emits the package:/build: keys that parse_recipe() reads back" is right about the key names and wrong about completeness. |
Add if self.install_args: data["install_args"] = list(self.install_args) alongside the other conditional fields. #124's implementation emits all twelve and round-trips exactly — I verified that one too — so if the restoration-vs-reimplementation argument is what is keeping this hunk here, #124 already has the correct behaviour and could simply be taken instead. Add the missing key to index_sync.py:335-347 in the same pass, or the field can never arrive. |
| 2 | Medium | core/eos/docs/three-way-alignment.md |
Reverting the vendored snapshot to the pin is the correct call and I am not disputing it — core/UPSTREAM.yaml says a vendored copy must match its pin, and the guard did its job. But the content being reverted away from is a correction: #109 replaced "25 YAML files / 25 board ports / ✅ Aligned" with "84 vs 83 vs 14/138 — three inventories describe the same set and nothing cross-checks them". After this merges, the repository again ships a table asserting ✅ Aligned for a set of inventories that #109 found do not agree. The body acknowledges this ("worth keeping; it belongs in ebuild's own docs or upstream in eos") but nothing in the diff or the PR carries it anywhere. A correction that is reverted with only a sentence in a PR body is a correction that is lost. |
Open the upstream eos issue or PR before merging this, and reference it from the diff — a line in ebuild's own docs, or the issue number in the CHANGELOG. The revert then reads as "moved", not "dropped". |
| 3 | Low | .github/workflows/scorecard.yml:27 |
ossf/scorecard-action@v2.4.3 is pinned by tag. A mutable tag is exactly what Scorecard's own Pinned-Dependencies check penalises, and this repo already knows the pattern — .github/workflows/linked-issue.yml on master pins its reusable workflow to @92cb596c773496ec4df76717e8acf0e6b7700f73. Raising the Scorecard job from the floor while leaving the action itself unpinned leaves points on the table in the same file. |
Pin to the commit SHA for the v2.4.3 tag, with # v2.4.3 trailing so the version stays readable. Same for actions/checkout@v4 and github/codeql-action/upload-sarif@v3 at :24 and :32 if you want the check fully satisfied. Out of scope if you would rather keep this PR to the four named repairs — say so and it stands. |
| 4 | Low | .gitattributes (new) |
*.yml/*.yaml text eol=lf is the right fix and the reasoning in the header comment is exactly right. One operational gap: .gitattributes governs future checkouts and commits, not files already in a contributor's working tree. Anyone with an existing Windows clone keeps CRLF locally until git add --renormalize . or a fresh checkout, so "it still fails for me" reports are likely. |
Add one line to the CHANGELOG or CONTRIBUTING telling Windows contributors to run git add --renormalize . once after pulling this. |
| 5 | Low | ebuild/packages/recipe.py, to_dict() |
The conditional-emission branches have no test in this PR — which is how finding 1 survives review, and is corroborated by the coverage bot's report on this PR. A single round-trip assertion would have caught it. | python\ndef test_to_dict_round_trips():\n r = PackageRecipe(name="d", version="1", url="https://e/x.tgz",\n build_system="cmake", install_args=["DESTDIR=/s"])\n assert parse_recipe(yaml.safe_load(yaml.safe_dump(r.to_dict()))) == r\n Fails today, passes with finding 1 fixed. |
Architecture conformance
Conforms. §21 places ebuild in Tier 1 — Foundation. Every code change is inside that repo, and the one file under core/ is a revert toward the pinned upstream, which strengthens rather than crosses a boundary. §5.1 is not engaged by any hunk: no import, link line or manifest entry changes direction, and eBuild understands the complete graph but is not a runtime dependency still holds.
The to_dict() restoration is the §23.2 "Package format — versioned .epkg/.eapp metadata schema" contract, and finding 1 is a conformance gap against it: the serializer does not emit the whole schema the parser accepts. The .gitattributes and Scorecard hunks are infrastructure, which §21 assigns to the Infrastructure tier and .github/STANDARDS.md §"Security" names OpenSSF Scorecard and CodeQL explicitly — moving a Scorecard job from permanently-red to green is direct compliance work under that section, and the body's evidence (gcr.io now requiring billing, v2.4.3 pulling from ghcr.io, eos already pinned there) is the kind of concrete justification STANDARDS.md §"Standards-compliance assertions" asks for.
No API break (brief item 8): to_dict() is a new method, purely additive; cast(Mapping[str, Any], ...) is a runtime no-op; the .gitattributes and workflow changes touch no interface. No performance concern (brief item 9). Documentation (brief item 11): the _PIC_FLAGS-style stale comment in plugins/__init__.py is corrected in place, and .gitattributes documents its own reasoning — but there is no CHANGELOG entry for a PR that changes five subsystems, which is inconsistent with #125, #126 and #131 in the same queue, all of which added one.
On duplication (brief item 10): the body is straight about it — three test hunks and the plugins/__init__.py hunk are stated as byte-identical to #122's, and I confirmed the blob hashes match (e08444f, 3c25922, b1d5e0f, 54605f0). That is the honest way to declare an overlap. It does still mean #122 and this PR cannot both merge meaningfully, and that to_dict() now has four competing implementations in flight — #119 (asdict, emits internal key names), #124 (all twelve keys, round-trips), #127 (conditional, includes install_args) and this one. Only #124 and #127 preserve install_args.
Proposed changes
- Fix finding 1 — one
if self.install_args:branch, plus the matching key atindex_sync.py:335-347. - Add the round-trip test from finding 5. It is the assertion that makes "restored verbatim from #111" a checkable claim rather than a provenance argument.
- Resolve finding 2 before merge: file the upstream correction somewhere durable and reference it.
- Decide the overlap with #122, #119, #124 and #127 as one decision rather than four. This PR plus #124's
to_dict()is the smallest combination that leaves master green and loses nothing. - Findings 3 and 4 are optional polish.
Not checked
- pytest — NOT RUN by me. pytest is not importable on this host, and the local
ebuildclone has a dirty working tree, left untouched per the rules of engagement. Unlike every other PR in this batch, this one does not need me to: nineTest (Python 3.x, os)legs are green on this head in CI, which is stronger evidence than a single local run. The body's680 passed, 1 skippedis consistent with that but I did not reproduce the count. - mypy — NOT RUN by me. Not installed. The body's "no issues in 107 source files" is unverified locally; the green
Testlegs imply theType check (mypy)step passed, sinceci.ymlruns it before the test step with nocontinue-on-error. - yamllint — NOT RUN by me. Not installed. The Windows CRLF diagnosis is the one part of this PR I could not check at all: I have no Windows runner, did not reproduce
core.autocrlf=truebehaviour, and did not confirm that the stray CR inauto-assign.ymlwas the specific file that broke thenew-lines: platformattempt. I am taking the body's account on that, and the greenTest (Python 3.x, windows-2022)legs are consistent with it. - Vendor drift — NOT RUN by me.
python scripts/check_vendor_drift.pywas not executed here;Compare core/ against pinned upstreamsis green in CI on this head. I did not independently verify that the reverted file matches blob7f9c8c1at eos pin5544c98. - Scorecard — NOT VERIFIED. I did not confirm that
v2.4.3pulls from ghcr.io, that gcr.io returns the billing error, or that eos's Scorecard job is green. All three are the body's claims. TheCodeQLandAnalyze (Python)checks green on this head do not cover the Scorecard workflow, which runs on a schedule. - The
existing-comments.txtfor this PR contains only the coverage bot; no human review points existed to avoid repeating.
Verified locally, against a git archive export of head f8cc3574:
ruff 0.16.5 check .→All checks passed!The four findings present onorigin/masterare cleared.to_dict()emits['build','dependencies','package','patches','url','version']for a recipe carryinginstall_args;parse_recipe(yaml.safe_load(yaml.safe_dump(...)))does not compare equal to the original, andinstall_argscomes back[]. Finding 1.- The four blobs the body cites as byte-identical to #122 match.
origin/masterstill carries all four ruff findings, so this PR's premise is current, not stale.
Automated architecture review of f8cc357421ac — scheduled, model claude-opus-5, checked against the EmbeddedOS Master Design v2.0. Advisory only: this reviewer never approves, requests changes, or merges. Reply here to discuss or push back — a wrong finding is a bug worth reporting.
Problem
CI — ebuildhas been red on master since the 09-08 batch merge, and because the first failing step isruff, everything behind it (mypy, pytest) has not run on master since.Vendored core driftandOSSF Scorecardare red on the same commit for unrelated reasons.Verified against runs 34419838440 (
CI — ebuildon8b623d5, all nine Test legs atLint (ruff)), 34419838427 (drift) and 34419838445 (Scorecard), and reproduced locally on master before every fix below.What broke
Lint (ruff, 4 findings). A duplicate
import shutil(F811), a lost trailing newline (W292), and twoimports two hundred lines downtest_ci_gate.py(E402) — that last one is my own #103, replayed onto a file that had moved. These three hunks are byte-identical to #122's (blobse08444f,3c25922,b1d5e0f), so whichever merges first leaves the other clean.Type check + tests (never reached on master since 09-08).
ebuild/packages/index_sync.py:354callsPackageRecipe.to_dict(). #111 defined it; #112, merged five minutes later from a base without it, deleted it in its replay. mypy names it once; pytest fails ninetest_index_synccases withAttributeError. The method is restored verbatim from #111 (cc90078): it emits thepackage:/build:keys thatparse_recipe()reads back, which anasdict()replacement would not — the YAML it writes has to round-trip through the same parser. #119 and #124 both propose re-implementations; this is a restoration of what master already had, not a third design.Two more steps that had never passed, found on this PR's first run (run 34591499237), once ruff let the job past its first step:
ebuild/plugins/__init__.py:46— those interpreters' stubs typeentry_points()as the deprecated mapping whose.get()wants anEntryPointsdefault (arg-type). The line carried# type: ignore[attr-defined], the wrong error code, so it suppressed nothing. Spelled out with acast, byte-identical to ci: fix the lint findings that stop CI before any test runs #122's hunk (blob54605f0).core.autocrlf=true, so every YAML file arrives CRLF and the defaultnew-lines: unixrule rejected every line oftemplates.yamlandlayers/eni/build.yaml. The step was added 09-03 and has never passed on that leg. A first attempt (new-lines: platform) failed the other way:auto-assign.ymlcarried one stray CR, so autocrlf left it alone and Windows then expected CRLF on its first line. Fixed properly:.gitattributespins*.yml/*.yamltoeol=lfso every OS lints the same bytes, yamllint keeps its defaultunixrule, and the stray CR is gone.Vendored core drift. #109 (
dba3d83) editedcore/eos/docs/three-way-alignment.md, a vendored copy pinned to eos5544c98, so drift went 44 → 45 and the guard failed exactly ascore/UPSTREAM.yamlsays it should. Reverted to the pinned content — verified the result is blob7f9c8c1, the same bytes aseos:docs/three-way-alignment.mdat the pin. The alignment note itself is worth keeping; it belongs in ebuild's own docs or upstream in eos, not in the snapshot.OSSF Scorecard.
ossf/scorecard-action@v2.4.0pullsgcr.io/openssf/scorecard-action, and gcr.io now refuses with "This API method requires billing to be enabled".v2.4.3pulls from ghcr.io; eos already pins it and its Scorecard job is green.Test plan
Verified, run the way
ci.ymlruns them:Not in this PR
EoSim Sanity's Windows and macOS legs install a wheel that has never been published (run 34561966328); #121 already replaces that with the clone the other legs use.