Skip to content

test: pin English per-unit boundary and dash regressions - #405

Merged
pengzhendong merged 1 commit into
masterfrom
codex/fix-english-rate-boundaries
Nov 6, 2026
Merged

pengzhendong merged 1 commit into
masterfrom
codex/fix-english-rate-boundaries

Conversation

@pengzhendong

Copy link
Copy Markdown
Member

Summary

English TN in the IndexTTS-pinned WeTextProcessing 1.0.4.1 misreads ordinary prose such as per day as a unit prefix and speaks a synthetic minus sign in valid per kg rates. Independent installs confirm that released 1.3.0 and current master already fix both behaviors. Although the unit-only expression still looks similar, sentence token boundaries and raw-field verbalization have changed.

Add 21 regressions for #403 rather than changing working production rule weights:

  • Ordinary per day/hour/second/month/minute/page/week/year/person prose must remain unchanged, with no partial measure classification.
  • Legitimate per kg/km/min, slash rates, and punctuation retain unit expansion without a spoken dash.
  • Cover the reported CI-system sentence, numeric rates, tag-to-verbalize behavior, n-best candidates, and exact input/output mappings.
  • Keep existing ambiguous unit readings, such as h, available without imposing a new preference order.

Verification

Installed PyPI versions into separate temporary target directories and ran them from outside the source checkout, rebuilding graphs (no old cached models). Both used Python 3.11 / Pynini 2.1.7 on macOS arm64:

Input 1.0.4.1 1.3.0 / master
It happens per day. It happens - per dalton y. It happens per day.
It happens per hour. It happens - per hour our. It happens per hour.
price per kg price - per kilogram price per kilogram
5 km per hour five kilometers - per hour our five kilometers per hour

The deployment resolution is to upgrade the pinned package to the verified 1.3.0 release, updating the consuming application's dependency lock as necessary. The token-boundary work in #365 and raw-field refactor 99dc9da explain why inspecting only unit_only_graph does not reflect the current pipeline behavior.

Tests

  • python -m pytest tn/english/test itn/english/test -q --tb=short: 676 passed, including all 21 new cases.
  • Changed-file flake8 and Git whitespace checks passed.

Related to #403. This PR adds regression coverage for an already-fixed issue; it does not introduce another grammar change.

@pengzhendong
pengzhendong merged commit 94ab845 into master Nov 6, 2026
1 check passed
@pengzhendong
pengzhendong deleted the codex/fix-english-rate-boundaries branch November 6, 2026 07:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant