Skip to content

Sweep the non-.rst prose for staleness - #339

Merged
derek73 merged 2 commits into
masterfrom
docs/staleness-sweep
Aug 7, 2026
Merged

Sweep the non-.rst prose for staleness#339
derek73 merged 2 commits into
masterfrom
docs/staleness-sweep

Conversation

@derek73

@derek73 derek73 commented Aug 7, 2026

Copy link
Copy Markdown
Owner

Summary

The last item on the 2.1 docs list: the places prose goes stale that grep over .rst misses. Four sites, two kinds of drift.

The Policy spellings didn't reach the docstrings

#334 moved the guide to the spellings that type-check, but only the hand-written .rst. Four _policy.py docstrings still taught the old ones, and autodoc renders them into modules.html — so the API reference and the guide disagreed on the same page.

site was now
_policy.py:566 Policy(maiden_delimiters={("(", ")")}) frozenset({("(", ")")})
_policy.py:587 script_orders={} script_orders=()
_policy.py:605 segment_scripts=() segment_scripts=frozenset()
_policy.py:625 maiden_delimiters={("(", ")")} frozenset({...})

Verified: grep for the old spellings in the rebuilt modules.html now returns 0.

The runtime warning was handing users an arg-type error

Sharper than the docstrings, because it reaches users who never open the docs. #337's new warning ends:

…or deactivate with Policy(segment_scripts=()).

A user pastes that verbatim, and in a package that ships py.typed it fails mypy. Now Policy(segment_scripts=frozenset()).

Nothing pinned this. The warning tests match on r"ja_segmenter" and never checked the deactivation hint, which is the actionable half of the message. Added a test asserting the offered spelling is the one that type-checks, and that the old one is absent.

Two stage headers understated their inputs

Checked mechanically — every stage's declared Reads: against the policy.* / lexicon.* attributes it actually touches — rather than by eye.

_script_segment, _tokenize, _assign, _group and _classify were already accurate. _vocab has no Reads: line by design, being a helper whose predicates take vocabulary as explicit parameters; its "no state" claim still holds.

Verification

  • uv run pytest -q — 3065 passed (was 3064; one new test)
  • uv run mypy, uv run ruff check — clean
  • sphinx-build -b html — exit 0, 0 warnings
  • sphinx-build -b doctest — exit 0
  • The corrected advice type-checks: a file containing all three new spellings passes mypy

🤖 Generated with Claude Code

Four sites, two kinds of drift.

#334 moved the guide to the Policy spellings that type-check, but only
the hand-written .rst. Four _policy.py docstrings teach the old ones
and autodoc renders them into modules.html, so the API reference and
the guide disagreed on the same page. Worse, #337's new runtime
warning hands the user Policy(segment_scripts=()) -- the library
telling you, in a package that ships py.typed, to write an arg-type
error. A test now pins the offered spelling; the warning tests matched
on 'ja_segmenter' and never checked the actionable half of the
message.

Two stage headers understated their inputs. _extract declared only the
two delimiter policy fields while reading three Lexicon suffix fields
through _suffix_shaped, which is not a detail: that is the mechanism
letting a clause's content overrule the delimiter, and #335 would
extend it. _post_rules omitted Policy.middle_as_family.

Checked mechanically rather than by eye: every stage's declared Reads
against the policy and lexicon attributes it actually touches.
_script_segment, _tokenize, _assign, _group and _classify were
already accurate.

_vocab has no Reads line by design, being a helper whose predicates
take vocabulary explicitly. Its 'no state' claim still holds.
@derek73 derek73 added this to the v2.1 milestone Aug 7, 2026
@derek73 derek73 added the docs Documentation fixes and updates label Aug 7, 2026
@codecov

codecov Bot commented Aug 7, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.48%. Comparing base (3c45594) to head (b6d71b2).

Additional details and impacted files
@@           Coverage Diff           @@
##           master     #339   +/-   ##
=======================================
  Coverage   98.48%   98.48%           
=======================================
  Files          41       41           
  Lines        2845     2845           
=======================================
  Hits         2802     2802           
  Misses         43       43           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Step 0 said 'review docs/ for anything stale' and named only .rst files
and AGENTS.md. That misses the three sites this sweep actually found
drift in, and it does not say the method: grepping for the changed
SYMBOL finds almost none of this, because prose describes behavior in
words rather than identifiers.

The sites are listed now, each one having gone stale at least once,
with the autodoc case called out for why an .rst-only sweep cannot
catch it. The stage-header check is given as a command rather than an
instruction to read carefully, since it is mechanical.
@derek73
derek73 merged commit 85dcc69 into master Aug 7, 2026
11 checks passed
@derek73
derek73 deleted the docs/staleness-sweep branch August 7, 2026 06:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs Documentation fixes and updates

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant